In the previous story, an old cache of XML documents brought back memories of a tough conversion project and made me wonder how the world has changed. It turns out that modern-day LLMs can make a real difference to such a project, but not always in a way that is obvious. If your documents are valuable enough to care about this kind of thing, you don't want any LLM making autonomous textual changes. But they can be used to enhance the environment in which decisions are made. And they can help humans focus their attention on things that matter most. If this had been available in 2015, the conversion project would have taken weeks, not months.
In this document we'll take a deeper look at some of the XML issues that are commonly encountered in such projects. Then, we'll describe how LLMs can contribute to a solution. It is time to show angle brackets.
If we step back a little bit, XML is all about adding markup to text. The markup declares what the text is meant to be; it structures and organises text, and it makes it possible to exchange ideas in a structured way. A schema is a contract that every instance should conform to. When all is well, a collection is not just schema-valid in the literal sense: its instances also follow the intention of the schema designer.
Most schemas evolve over time. This could be maintenance, reordering to improve processing, a change in the underlying information, or an external standardisation effort. In 2015, the 250K XML collection needed conversion from one schema to another. Even though the information was not supposed to change, this was a substantial design effort. The common way to do this is to write one or more XSLT transformations. XSLT is optimised for working with XML documents. It is elegant and powerful, but most definitely an acquired taste. As a general rule, it makes a lot of sense to do some housekeeping on the source documents before trying to handle every exception inside the XSLT code. This is especially true when the input quality varies, and that was definitely the case here.
So what kind of housekeeping are we talking about? There are three categories: errors and inconsistencies in metadata, wrong choice of elements, and redundant markup. Notice that we're not talking about textual corrections. Even in cases where the text was obviously wrong, we would still not correct it. This was a legal requirement as it would have to be formally republished after a textual change.
Metadata sits around the text rather than in it, so a metadata edit is not a textual change and does not necessarily trigger republication. That does not make it free: a title in the metadata still has to match the title in the document, and a value still has to come from whatever vocabulary defines it. Fewer constraints than the text, not none.
The conversion of metadata is defined by mappings. Key-value pair X in source becomes key-value pair Y in target. If it is just a different name for the key, then the mapping is easy. If the value needs to change then it gets harder. The latter happens, for example, when a controlled vocabulary (which defines the values) changes. A one-to-one mapping of controlled vocabulary values is generally easy. But sometimes values must be shoehorned into new categories or split into something more detailed. Another common issue is a change of data type. For example, suppose that the source stores several values in one delimited string. If the target uses an explicit list structure, that string needs to be split into elements.
In practice, this means that metadata conversion can be handled mostly by standard, deterministic Python or XSLT. But there is one specific area where LLMs can be a timesaver. The shoehorning or translation of values is something that an LLM excels at. Notice that this still requires a well-considered prompt: just asking for a synonym is not going to be helpful.
The pattern is clear: handle as much as possible with deterministic code, and let an LLM handle the remainder. In fact, the LLM can also help with writing the deterministic code. Feed it an input and an output, and it will write code for you, which you can then test and refine. This even holds true for value translation. Based on an analysis of the patterns in the metadata, the LLM can write a set of rules that are then executed by a Python or XSLT program (for example: "[Bb]anking becomes Financial Services"). If the value is genuinely free text, it's probably better to store the original alongside the mapped one, instead of discarding the original.
Especially in document-oriented domains, lots of ink was spilled on the difference between the meaning of text and its presentation. The aim was to decouple presentation, which would allow for different views of the same underlying text. In practice, this didn't always work out, because of limited tools or imagination. The 250K document collection was authored by many different people and organisations, and this made things worse because there was no consistency at all. The schema was very much in favour of coding the meaning, but the end result would be a PDF, so many authors just took a shortcut.
The following XML is an example of what happened:
<al>De gemeente bepaalt algemene en bijzondere</al>
<al>voorwaarden die van toepassing zijn.</al>
Even though the al element is supposed to wrap around an entire paragraph, in this case each line is wrapped with it. It is like an old typewriter: the end of the line, rather than the end of the thought, decides where the markup stops. This is typewriter coding.
Another example of wrong element choice is the following fragment:
<al> In deze verordening wordt verstaan onder:</al>
<al>
a. markt: de door het college ingestelde warenmarkt met uitzondering van de jaarmarkten en halfjaarlijkse markten;
</al>
<al>
b. standplaats: de ruimte die voor de duur van de markt is aangewezen voor het uitoefenen van de markthandel;
</al>
This is a definition list with two items a. and b. It should have been coded as a lijst element with li children. Again, it is an example of literal "typewriter coding".
And sometimes the use of whitespace would be even worse:
<slotformulering>
<al>Secretaris, Burgemeester,</al>
<al> </al>
<al>Dit besluit is bekendgemaakt op: 17 juni 2004</al>
</slotformulering>
There are specific elements in the schema for the roles ("Secretaris", "Burgemeester"), but instead it is coded with meaningful whitespace.
One could argue that these examples don't matter when doing a schema conversion, and strictly speaking one would be right. The target schema has a similar al element so anything converted directly would still validate. However, there are several reasons for not letting this sit. First, the original coding is arguably wrong. One of the hard requirements for this particular document collection was accessibility. Screen readers and other assistive tools work better when there is a clean coding of meaning instead of presentation. Also, this was around the time the PC, with its big screen, stopped being the primary channel. Typewriter coding is notoriously hard to reflow or reformat on different devices. But ultimately, it turned out that keeping this markup unchanged made the new PDF production pipeline unwieldy. The old production pipeline had accumulated many implicit choices that quietly compensated for coding misuse. The new pipeline didn't share them, so it kept throwing up strange differences. At that point, you either keep tweaking the display engine or fix the underlying XML markup.
The problem in 2015 was that fixing the markup would require manual work. With deterministic code, it is relatively easy to detect and fix these cases, but it is really hard to cater for all exceptions. For example, there are legitimate cases where two al elements should not be merged, because the author may have forgotten a full stop, and merging would hide an error. A human would spot this immediately, but it is quite error-prone to capture this in a regular expression because it would have side effects in other cases.
Using an LLM changes the equation. Now we can be more aggressive in changing occurrences with Python or XSLT, and use the LLM to flag whether the change altered the meaning. Just like with the metadata, the LLM is not used to recode, but it can be used to advise.
Editing XML was always a source of confusion for authors. Unlike Word, XML editing does not fit easily in the WYSIWYG (what you see is what you get) paradigm. The XML source view with elements, attributes and text was generally considered too unfriendly for authors, so there were many different editing tools, for example using forms or other means to guide the user without exposing the dirty underbelly of XML.
Using tools was generally fine, but sometimes the artefacts of using them became visible in the output they generated. Take for example this fragment:
<al>
<vet>H</vet>
<vet>E</vet>
<vet>F</vet>
<vet>F</vet>
<vet>I</vet>
<vet>N</vet>
<vet>G</vet>
</al>
For whatever reason, the tool has wrapped each character with a vet (bold) element. It would have been better to merge them and use <vet>HEFFING</vet>.
Again, one could argue that this is harmless, but it is ugly and could have negative consequences downstream. For a human reader, it is obvious that these can be merged into one, and in nine out of ten cases it wouldn't even be complicated to do so in Python or XSLT. However, and this is the key point, there are cases where merging elements must be done more carefully. For example, if there is a comma halfway through (for example <vet>F</vet>,), then the merge should stop before the comma and restart after it. Meaningful whitespace leads to similar constraints. Because we have the before-and-after XML fragments, we can use an LLM to check if the merge is OK. In this case, we don't even need the XML tags. We can just ask: "Is 'HEFFING' one word?" If the answer is yes, we're good. Otherwise, we will have to look again.
The three examples above are interesting, because individually they are quite manageable and easy to understand. The problem is when a large content collection has 100s of these issues. It is not just a real pain to manage each individual issue, but the interaction between issues might also make life more complicated.
Even worse, each example has plenty of potential for exceptions that are hard to spot with deterministic code. And the exceptions do not arrive on schedule. You write code after looking at a few hundred documents. The exception turns up in document number 16352, weeks later, and often after the code has already run. Now the challenge becomes how to find those exceptions and redo the documents that were processed with the "ignorant" code. Good luck doing this when you're working on a Windows shared network drive.
Every data conversion project struggles with exceptions. XML makes them harder to catch, for three reasons.
First, the default XML safety mechanism (schemas) doesn't catch any of this. Every fragment in this document is schema-valid. Validation tells you that a document follows the rules of the schema, not that it follows the intention behind them. There is a grey area: a gap between valid and right.
Second, in document-oriented XML the text and markup are woven together. Markup sits in the middle of sentences, so a change to the markup happens right next to the words. A comma, a space or a line break can carry meaning, like the whitespace between "Secretaris," and "Burgemeester,". A tool that tidies up whitespace, or a merge that pulls a comma into a bold element, changes what the document says while the text looks almost the same.
Third, every fix changes the ground for the next one. A transform finds its cases by their structure: a run of al elements, a run of vet elements. Once one fix has run, that structure is different. A rule written against the old structure may now find something else, or nothing at all. Fixes interact, and the order in which they run matters.
In 2015, it was a common strategy to work with large to enormous XSLTs to do conversion. Every rule and every exception went into the same pipeline, and every late discovery made it more complicated. Change one part, and something else silently breaks.
Even then, there was a drive to work in a more granular fashion. For example, XProc tried to integrate XSLT into pipelines. Although conceptually sound, this never really broke through. One reason was the lack of mature implementations. But the other reason was that XProc didn't solve the real problem. It split the transformation into steps, but a step is not an issue: it has no rule of its own, no exceptions and no test.
Document conversion is about management of issues. And the smaller and more focused each issue, the better we can describe it and develop a solution. Each issue has its own rule, its own exceptions and its own test. When something goes wrong, you know which issue did it.
This is also exactly where LLMs are strong. Ask a model to convert a whole document, and it will get almost there but not quite, and finding where it went wrong is as much work as doing it yourself. But ask it whether "HEFFING" is one word, and the question is easy to answer and the answer is easy to check. The smaller and more clearly delimited the problem, the better the model performs. The same is true for people: an issue that fits on one screen can be understood, tested and approved.
So this is the key to conversion and upcycling at scale: many small issues, each clearly delimited, instead of one big transformation.
Working with small issues comes at a price. In any non-trivial collection you will end up with hundreds of them, running in order, with exceptions that force you to go back. All of that needs bookkeeping. Somebody has to keep track of which issue changed which document, and who approved it. Small issues only work on a good foundation, and that foundation is the subject of the next document.
Do you hold a large XML collection, and have good reasons to put AI to work on it, but no safe way to begin? Here is how we would work together. It starts with an hour online, no charge: we look at your own documents, and you see the system running on them.