How to Validate AI-Suggested Reference URLs Automatically
What you’ll learn
- Why AI-suggested references should be checked for URL availability before human review
- The basic structure of a URL-checking script and how to classify its results
- How to separate accessibility from evidence accuracy and choose the right execution point
Check Reference URL Availability Before Evaluating Evidence
Reference review becomes more efficient when machines check whether a URL exists before people read the source. On this blog, plausible-looking candidate URLs sometimes pointed to pages that did not exist. The check classifies reachable, missing, authenticated, timed-out, and redirected results, while people separately judge whether the content supports the claim.
By the end of this article, you will have practical criteria for answering “How can missing reference URLs be classified separately from temporary access failures?” in your own context.
AI can suggest reference URLs that do not exist
When I have AI suggest reference candidates for an article, it produces URLs that look correct at first glance. But when I actually try to visit those URLs, some of them turn out to be pages that do not exist.
This is a characteristic of how AI works. AI has learned what URL formats look like, so it can suggest plausible-looking URLs without checking whether they are actually accessible.
To address this, I built a script that automatically verifies whether reference URLs in articles can actually be accessed.
This article is limited to mechanical URL availability checks. I cover citation decisions and whether a source supports its claim in the twelve-step review workflow.
The URL Checker Extracts Reference Links, Tests Responses, and Classifies Results
The URL-checking script runs the following steps.
Step 1: Extract URLs
The script reads each article’s Markdown file and pulls out URLs from the references section (## 参考文献 or ## References).
Step 2: Check access
The script sends an actual HTTP request to each URL and checks the response. Think of it as the script doing what a person would do when opening a URL in a browser — except it does it automatically for every URL in the list.
Step 3: Classify results
The script classifies results based on the response.
Classify URL Results as Reachable, Missing, Restricted, Timed Out, or Redirected
Accessible (normal)
An HTTP status in the 200 range means the page returned successfully.
404 (page does not exist)
A 404 response means “this page does not exist.” A URL returning 404 cannot be used as a reference, so it is treated as a Critical issue. The reference must be removed or replaced with a correct URL.
Authentication error (login required)
A 403-range status indicates that a login is required to access the page. For a published article’s references, readers may not be able to access such a page, so replacing it with a different source is worth considering.
Timeout
No response within a set wait time. This can be caused by a temporary network issue, so the script does not immediately flag it as Critical — instead it is marked for rechecking after some time.
Redirect
The URL responds but forwards to a different URL. If the destination is appropriate, there is no problem, but it is recommended to update the article to use the canonical destination URL directly.
An important caveat: accessible does not mean accurate
The URL-checking script confirms “can this URL be accessed” — it does not confirm “does the content at this URL support the claim in the article.”
A URL may be accessible but contain content that is not appropriate as a reference. Also, a page’s content may have been updated since the article was written, meaning the source no longer says what it once did.
The script checks URL reachability. A person still checks whether the source supports the article’s claim, whether it is appropriate as primary information, and whether the information is current.
Run URL Verification Before Publication
On this site, I run the URL check with npm run review:references:live. Running this script is part of the review process before publishing any article.
Because this script makes requests to external URLs, a network connection is required. To avoid placing unnecessary load on external servers, the script waits between requests rather than sending them all at once.
Summary: Automate URL Availability Checks but Keep Evidence Fit a Human Decision
Reference URL candidates suggested by AI can include links to pages that do not exist. Using a URL-checking script makes it possible to detect broken links before an article is published. However, verifying that a URL is accessible and verifying that a source is appropriate are two separate tasks. Combining automated checks with human review is what makes the overall process reliable.
Start by checking one reference candidate and separate confirmed 404 or 410 results from authentication, rate limits, and temporary failures. A reachable URL still does not prove that the source version or content supports the claim, so automated availability cannot approve a reference.
This article is a general information summary and is not legal advice. Confirm practical decisions with a qualified specialist.