Reverse Link Disabled

When you click on the title of a page, you get a list of all of the pages that reference that title. It used to work, then it was disabled, and finally it is working again. What follows is a discussion and history of this feature. See BackLink, ReverseLinkEnabled.

Normally the title of a page links to its citations. As implemented, this is one of Wiki's more expensive services. Unfortunately, Wiki is being swamped by "not too bright" robots that pound away at these searches. I'm forced to disable this feature until I can handle the load or offer an alternative to cloners. -- WardCunningham

Suppose we no longer support the hyperlink from the document title, and replaced it with a button. The button triggers a cgi POST operation that initiates the full search.

This would mean that the search doesn't appear as "just another link". Is it safe to presume that crawlers never initiate POST operations on Web servers? - if so crawlers just GET pages and users can conveniently click on the search buttons. -- DafyddRees

This is a good idea and is part of my newer designs. -- WardCunningham

Another option, if the "click the title" interface is desirable, is a simple javascript addition to the page that turns the (previously unassuming) title into a reverse link. It's generally safe to assume that bots don't run javascript. You could even include a POST button that gets removed by the script -- this way you don't "leave out" users who disable scripting, but still allow for a more elegant user interface for everyone else.

I've been rejected from this site with the "you are dominating me!" message. That's not necessarily a bad thing, but you should know that you're catching humans as well as robots. I wasn't doing anything special; that is, I wasn't trying to break it, but I was flipping through pages fairly rapidly. (Sometimes a series of pages appear in RecentChanges with no change that I can see.) -- DaveHarris

The message is actually Can't sustain current request rate from site .... It is automatically issued to four or five sites a week. Most of them grind away retrieving the error message. I just denied 1548 requests from 212.22.70.4. I'm sorry I caught you too. The condition does clear quickly. Please forgive the inconvenience. -- WardCunningham

Reverse links are no longer disabled. I've done two things to make this possible. First, I've made a full search for a link name run in a few seconds. This is still slow enough that a spider program could swamp my server if it ran more than a few dozen of them. So, second, I've made the search URL be the same for all pages. A spider won't want to run more than one search if it can't figure out how they would be different. I use the "referer" property for the search key. This little trickery will fail for people who disable this in their browser, but the rest of us will have reverse links again. Thanks for your patience. -- WardCunningham (11/23/99)

CategoryWiki