The hardest problem in fieldwork isn't the fieldwork

Most problems in software are wider than they are deep. The job is usually plugging and wiring things together, or shovelling data from A to B: breadth, not depth. It’s historically been the driver of engineers’ collective groan at data-structures-and-algorithms interview processes, when does anyone actually use this stuff? But on the rare occasion you do come across a real DS&A problem, it’s a treat, for me anyway.

Here it is, in plain terms.

We help organisations in water, energy and financial services carry out fieldwork at scale: visits to addresses, often under a warrant, that have to be done by the right people, in person, and inside a tight legal window. Imagine a team with a van and an eight-hour day. In front of them is a list of hundreds, sometimes thousands, of addresses they could visit. The question sounds simple:

Which visits should this team do today, and in what order?

It isn’t simple. It’s probably one of the hardest questions we deal with.

Why it’s hard

Start with just the order. Once you’ve picked a dozen addresses, finding the shortest route that visits them all is the Travelling Salesman Problem. It’s one of the most studied problems in computer science, because it’s much nastier than it looks. The number of possible routes grows factorially as you add stops, so it gets out of hand fast. Twenty addresses can be put in order in more than two quintillion ways. If you could check a billion of them a second, you’d still be going the best part of a century later.

And that’s the easy half.

We can’t get through everything in a day, though. There’s almost always more work than there are hours for it, and not every case carries the same priority or the same value to the client. So the real question isn’t whether a case gets visited, it’s which cases a team takes on a given day, and in what order. Nothing gets dropped; it’s a matter of sequencing the work so the most pressing and most valuable cases come first.

That changes the problem. It’s no longer plain Travelling Salesman, it’s one of the vehicle routing problems: specifically a prize-collecting route, closer to what’s called the orienteering problem. The goal shifts from finding the most-efficient way through a fixed list to spending a finite day well, so the most valuable and most pressing work gets done first. And “value” isn’t one number. It’s a trade-off between things that pull in different directions:

• the priority of the case
• how close it is to a hard legal deadline (a warrant only lasts so long, and if it lapses you start the whole thing again)
• how tightly the day’s work clusters geographically
• the value of the case to the client

and a few more besides. Push hard on any one of them and the plan gets worse somewhere else.

Warrant work adds another layer on top of that. A single visit can require three different field operatives, from three different companies, at the same address at the same time: a warrant officer, a locksmith, and an engineer to carry out the work itself. So the job stops being one team’s day and becomes a scheduling problem across three separate organisations, all against the same route on the same date. If any one of them can’t make it, the visit can’t go ahead.

on today's run high value / urgent scheduled for later

On top of that there are hard rules you simply can’t break: regulated start and finish times, contractual working hours, the client’s own processes and policies, HSE protections. Break any one of them and the plan isn’t just worse, it’s invalid.

So you’ve got to choose the right subset for a given day from cases with several competing notions of priority, stay inside the rules, and then solve a famously hard ordering problem on whatever you chose. And not once: again and again, because the moment a case settles or cancels the best answer shifts and you want to replan. Across a whole region. Several times a day.

That’s the part I got properly excited about. It’s a lovely problem to chew on.

What we built (and what I’ll keep to myself)

I’ll keep the inner workings to myself, but the general shape I’m happy to share.

We built a planning engine that knows nothing about warrants, or water, or energy. It understands the abstract problem (value, time, distance, eligibility, hard limits), and the domain-specific bits are fed in as configuration. That matters more than it sounds. The same engine can plan a smart meter installation campaign, a warrant enforcement round and a maintenance schedule all at once, each with its own rules, and those rules can be tuned by the people who understand the work rather than rewritten by engineers every time something changes.

It isn’t tied to a fixed idea of what a good plan looks like, either. We can give it any number of targets we like and weight them to match what a particular customer actually values, rather than baking in one notion of “good” and making everyone live with it.

It plans against real road travel times rather than straight-line distances, which counts for a lot once there are rivers, motorways and one-way systems in the way. It finds the best possible order for each run, not an approximation of it. And it doesn’t just produce an answer and walk off: it puts a proposed plan in front of a person, with its reasoning attached, and waits for a yes or no before anything is committed. The computer does the heavy arithmetic and a human stays in charge of the decision.

The bit I’m most pleased with is the bit you’ll never see. Doing this once, for a dozen stops, is a student exercise. Doing it across thousands of cases, replanned many times a day, fast enough that nobody’s sat watching a loading spinner: that’s where the actual engineering is, and that’s the bit that stays in the cupboard.

Why this matters to you

If you’re a client, the short version is that you get more of the right work done in a day for the same cost, and the cases running up against a legal deadline don’t get quietly missed.

If you’re one of our field agencies, it means the work that reaches you turns up already arranged into sensible, geographically coherent days, instead of a pile of addresses you have to sort out yourselves.

And for me, it’s a reminder of why a formal computer science education isn’t necessarily a waste of time in this profession. It can give you the tools to recognise the formal shape of certain problem classes, an intuition that has become supremely powerful in the age of AI.