Possible to remove in-text references?

[alimoges42]

Hello, I recently started using Docling but so far have been unable to find a way to remove in-text references. For example, if I have the below section of text, is it possible to remove the numeric references? I'm planning to use this primarily on academic papers, so simply filtering out all numeric values using something like regex isn't an option, as I'd like to keep any relevant statistics and other numeric data intact.

Sample text:
"We found that treatment A was associated with increases in outcome X (p<0.05) as well as a 45% reduction in serum levels of marker Y (p<0.05). This further supports the work of Smith et al. (2025) in identifying this crucial pathway 16 ."

If the "16" at the end is an in-text reference to the theoretical Smith et al. paper, is it possible to remove the 16 from the text while leaving all the other numbers intact? If this is not currently possible within Docling, would anyone know of a good workaround?

Thanks in advance for any help!

[alimoges42]

For anyone interested, I was able to use regex after all. I've tested it on a few examples for flexibility and it seems to work well. Code is below.

import re


def filter_numeric_citations(text: str) -> str:
    # Pattern for matching different citation formats
    citation_patterns = [
        # Single numbers or lists within parentheses/brackets
        r'\s*\((?:\d+(?:\s*,\s*\d+)*)\)(?=[\s\.,])',
        r'\s*\[(?:\d+(?:\s*,\s*\d+)*)\](?=[\s\.,])',

        # Ranges within parentheses/brackets
        r'\s*\((?:\d+\s*-\s*\d+)\)(?=[\s\.,])',
        r'\s*\[(?:\d+\s*-\s*\d+)\](?=[\s\.,])',

        # Mixed lists and ranges within parentheses/brackets
        r'\s*\((?:\d+(?:\s*-\s*\d+)?(?:\s*,\s*\d+(?:\s*-\s*\d+)?)*)\)(?=[\s\.,])',
        r'\s*\[(?:\d+(?:\s*-\s*\d+)?(?:\s*,\s*\d+(?:\s*-\s*\d+)?)*)\](?=[\s\.,])',

        # Standalone citation numbers (more restrictive)
        r'(?<=[^\d%$\.°])(?<!\.)\s+\d+(?:\s*,\s*\d+)*(?![%\.°\d]|\s*[a-zA-Z])(?=[\s\.,])',

        # Author citation format
        r'\s*\([^()]*?\d+\)(?=[\s\.,])'  # Matches "(Smith et al. 15)" format
    ]

    # Combine all patterns
    combined_pattern = '|'.join(citation_patterns)

    # Replace citations with empty string, preserving necessary spacing
    cleaned_text = re.sub(combined_pattern, lambda m: ' ' if m.group().strip().endswith(').') else '', text)

    # Clean up any double spaces and space before punctuation
    cleaned_text = re.sub(r'\s+', ' ', cleaned_text)
    cleaned_text = re.sub(r'\s+([.,])', r'\1', cleaned_text)

    return cleaned_text.strip()'

Testing functionality

test_cases = [
    "The study showed a 15.7% increase in efficiency [15, 17-20].",
    "Previous research (15) demonstrated that 42% of cases...",
    "Smith et al. [15-18] found that 250 participants...",
    "The system achieved 99.9% accuracy (15, 17, 19-22) in trials.",
    "Temperature increased by 15°C (Smith et al. 15).",
    "Distance traveled decreased by 20cm (Smith et al. 12)."
]

for test in test_cases:
    print(f"Original: {test}")
    print(f"Cleaned:  {filter_numeric_citations(test)}\n")