Content Distribution Network (CDN)
Sometimes this function is activated on the server or you can activate it yourself.
The CDN collects the data that belongs to a web page that is on it's way to the first user and stores it close by.
When another user in the same area needs the same web page, it is sent instantaneously to that user.
A web server then only needs to be informed that the same page has been sent to another person, but it will not need to send it itself.
That saves duplicate long distance data transmission.
It is not defined whether or not search engines have been included in the CDN functions and operations. They use to have to and will probably still have to go to each specific server in order to get the latest information for themselves, because the content of the robots.txt and sitemap.xml files are not requested by web browsers and would therefore not be stored by CDM server function.
It is not documented which file transport function search engines use in order to get access to web pages and their associated data.
With or without CDN, there is web page caching being done by anyone who is in a position to do it.
It is only a question of how long those pages are kept.
Over the years, search engines can get internet address and web pages mixed up.
Further more, search engines have been known to try and look for old and none existent pages.
The sitemap.xml and robots.txt files are designed to help alleviate the problem. Keep them updated and hope that search engines only use their content.
CDN will inform a server when it has sent a page in it's stead, but there is no documentation or clear statement about whether or not search engines are also informed or if their hard -and software is capable of receiving and evaluating such messages.
CDN does not seem to attach any statements and opinions to data, except when it informs servers that it will or wants to sent a page for them.
Use CDN to bring your
html content closer to surfers in any part of the world