Why LLMs Didn't Cut It

If this article interests you, consider reaching out. We’re actively looking for ambitious researchers to join our mission. This article is for a general curious audience.

We began Columbus with the seemingly simple goal of applying AI to maps. To create our MVP, we first tried the solution everybody else uses. We thought it would be fast and straightforward.

Our first product version was called MapsGPT. Putting a large language model in front of well-formatted geospatial data and letting people ask questions about places. Though the platform gained traction, with thousands of queries within the first month, we found it to be effectively impossible to use reliably and to scale to critical business use-cases. Even an advanced LLM wired to live tools through an MCP server consistently failed to satisfy our benchmarks or edge cases. Prompt-engineering could not make up for the fact that LLMs were not built for geospatial products.

Here are the inherent problems with using LLMs for geospatial AI that Columbus is working to solve. We observed these problems in the real world across thousands of queries with an LLM from live-users on MapsGPT:

1. No native understanding of coordinates or physical space.

An LLM tokenizes everything, including coordinates. A latitude/longitude pair becomes a sequence of sub-word tokens, and the model learns statistical patterns over those tokens. That's a great way to handle language but not specially designed to handle a continuous, three-dimensional surface where two points that are textually similar can be physically far apart, and vice versa. Numeric and spatial reasoning is a well-documented weak spot for token-based models.

2. Missing live, physical, and anthropological context.

LLMs are only trained on text that was written down publicly and structured. Though not an inherent problem with LLM architecture, our architecture requires specialized geospatial data and more of it. Enormous amounts of what's true about a place were never written, recorded, or are somewhere the model never saw, for instance, municipal surveys, user-generated content, out-of-date maps, or a malformed CSV. The model doesn't have the data, so it can't reason about it, hence our technology: Data Enzymes.

3. Limited semantic reasoning for geospatial context.

LLMs lack native geospatial representations and must reason about space indirectly through text. We tried an extensive number of retail LLMs, but none had an authentic semantic core reasoning on geospatial data. No model that consistently understands the fuzzy-but-clear accurate meaning of whatever we're looking at or asking for, especially in edge cases.

4. Hallucination and long-context degradation.

Modern LLMs can reason effectively over large contexts, but critical reliability remains an open research problem. Hallucinations can arise from gaps in training data, imperfect retrieval, weak grounding, or failures during long-chain reasoning. Long-context settings introduce additional challenges, including attention dilution and difficulty maintaining focus on the most relevant information.

These issues are particularly important in geospatial applications. Hallucination is not unique to long contexts, but long contexts often make reliability harder to maintain. For geospatial work, where a mistaken coordinate, boundary, or attribute can materially change a result, even small errors can make a plausible-sounding answer factually wrong.

5. A bias toward the well-trodden.

Because of how embeddings and training data work, an LLM's answers gravitate towards the mathematical mode, the popular and highest-probability region in its training data. Ask it about a place that is data-rich in its knowledge, and it does fine; but ask it about a place that is data-poor or unconventional, and it pattern-matches to the nearest popular thing.

6. Complexity factor: The ‘Context Window Trap’ (why a city isn’t the same data as a book)

Another fundamental bottleneck of applying LLMs to geography is the reliance on the context window itself. One might ask: why would an LLM need a multi-million-token context window just to analyze a specific piece of land? The answer reveals the core architectural flaw: LLMs force us to serialize 3D, interconnected space into a 1D linear string of text. To make a language model "see" an entire region at once, you are forced to translate every coordinate, building footprint, sensor reading, and demographic statistic into millions of sequential words. Even with an infinite context window, the model is still just reading a massively long, diluted list. It fundamentally lacks native geometric or topological awareness. This is why scaling LLM context windows does not equal scaling spatial intelligence. A native LGM bypasses the token-context bottleneck entirely. It doesn’t "read" a city sequentially in a context window; it represents it as a region natively, as multi-dimensional, structured, interconnected spatial data.

To be clear, LLMs are useful to us. As later posts explain, language-model machinery is excellent at understanding what a human is actually asking, and we need it for exactly that. And though LLMs are incredible and making progress, why keep trying to use an LLM for something it is not designed for? In the core task of reasoning over Earth, we could not find an alternative. So, we are building it.

We're hiring researchers who find this topic as interesting as we do. Join us.