Deep Linking

The practice of accessing a Web document directly via a (often convoluted) URL (often containing oodles of CommonGatewayInterface stuff). Often done against the wishes of the Web site publisher, who would prefer user to navigate to any content via the links exposed on the front page of the site (and their transitive closure).

Often disliked by Web site publishers because:

Bypasses advertisements; publishers are paid by (among other things) the number of users that gaze upon the ads. (Clicking on the ads can increase the publisher's revenue further).
Might bypass other legalese.
For an incompetently-run site, might even allow access to material available on a for-payment basis only. (Competent Web publishers with "premium" content do know how to block DeepLinking). Actually, the problem here isn't deep linking, but access control. It should be possible for authenticated users to reach content through a deep link. It's desirable for unauthenticated users to be prompted to authenticate when requesting controlled content, and then forwarded to that content, no matter how deep the link, when the authentication is successful.

Ways to prevent it:

Automatically rename documents periodically; so a given hard-coded URL is not valid for very long.
Use cookies as a condition for accessing the content.
Use NagScreen?s. (The UserFriendly page has a fine example of such. Seems to be gone, got another example?)
Configure your Web server so that "deep" pages are only accessible if the REFERER header is your site -- if not, bounce them to your home page.

Hiding deep content is a trade-off. You could make it almost impossible for non-privileged visitors to access deep content, but it's usually not worth the bother. For example, the New York Times' Web site moves old articles to a pay-for-access archive, but workarounds that allow free access are widely known. I'm sure the Times is aware of them, and assume they've decided that a more restricted access system isn't worth the price of a little freeloading.

Moved from KeepYourHeadDown:

(Wayne...) I appreciate your editing the above hyperlink to make it easier for but :-) I have attempted to route everything through the front door of my website. I want to be able to rename app.asp or the "bloops" link without breaking any pointers to it. I think there are enough broken links on the Internet to keep us busy editing pages for years.

I understand, but how do you keep people you don't know from creating links to your internal pages? But I'm happy to change it back (I just did).

I once wrote a script which redirected anyone not coming in from the front page back to the front page. Another solution to the broken link problem would be to make his Error 404 page point to the front page so they could find it that way. Alternatively you could set up a script to do a search on the site for the page not found; as his done on http://php.net/thisprobablywontbefound.html -- MatthewTheobalds

The auto-search idea is a neat solution to this problem.

The idea that anyone can link to anything is a key part of what makes the web work. You can't stop people from doing it, but you can certainly discourage people from visiting your site by moving pages and then forcing the user to dig for them from the top page. If you move a page, consider providing the web equivalent of what programmers call a forwarding method. You can leave a stub page behind, you can do the http "moved" thing, or in Apache, you can use URL rewriting to automagically redirect the user to the new page.

I agree, leaving a Stub page behind -- as, incidentally, is done on Wiki when a page is refactored and moved -- is probably the most elegant solution to the problem. In conjunction with a search facility, one could get a pretty tight Error system. -- MatthewTheobalds

Just so we are clear, "the idea that anyone can link to anything" is not a key part of anything (the web included.) People "can" link to things that remain on websites (privately paid for and subject to change for economic (and other) reasons but they can not now and will never be able to "link to anything.")

Clearly forcing people to dig isn't a good idea and nobody suggested doing so. -- tl

People have the technical ability to link to any web pages that they visit. This is indeed a key idea on the net -- if linking were restricted to only top pages, the web would be much less useful.

As you point out, the webmaster can always make these interior links useless. That is usually a bad idea, unless the webmaster's goal is to frustrate the site's users.

And if this were to happen "What Users?"

Of course if linking were "restricted" to "only the top pages" the web "might" be less useful. But then neither I nor anybody I know suggested such a thing. I'm quite certain I said that I like my site to be that way.

That everybody can think of "an alternative" is obvious. Follow Usenet News or the WikiWikiWeb and you will find a million experts on every subject. -- tl

I apologize if I gave you an impression of criticism; this was never my intention. I aimed to offer a solution to the problem of broken links - not to cause an argument. Please feel free to delete any of my comments which offended you and will not be of use to others. -- MatthewTheobalds

See also DeepLink on MeatBall for more discussion.