text:
|Numéro de publication
|Type de publication
|Numéro de demande
|Date de publication
|25 sept. 2007
|Date de dépôt
|1 mars 2002
|Date de priorité
|1 mars 2001
|État de paiement des frais
|Autre référence de publication
|Numéro de publication
|Claudson F. Bornstein, Timothy K. Canfield, Gary L. Miller, Satish B. Rao, Ravi Sundaram
|Cessionnaire d'origine
|Akamai Technologies, Inc.
|Exporter la citation
|BiBTeX, EndNote, RefMan
|Citations de brevets (8), Citations hors brevets (9), Référencé par (115), Classifications (31), Événements juridiques (4)
|Liens externes: USPTO, Cession USPTO, Espacenet
This application is based on and claims priority from Provisional Application Ser. No. 60/273,241, filed Mar. 1, 2001.
The present invention relates generally to finding the best way to route messages between various locations across the Internet while quickly adjusting to changing network conditions.
Optimal online performance has become a paramount focus for enterprises seeking to deliver a rich, interactive user experience. As more and more of today's business transactions are conducted via the Internet, companies must make certain that transactions are executed without interruption. Whether it is a purchase form, an online reservation, a customer survey or an information update, the enterprise must have some way to ensure that the transaction is complete.
One way that enterprises have sought to manage and move information effectively via the Internet is through the use of content delivery networks. A content delivery network or "CDN" is a network of geographically distributed content delivery nodes that are arranged for efficient delivery of content on behalf of third party content providers. A request from a requesting end user for given content is directed to a "best" replica, where "best" usually means that the item is served to the client quickly compared to the time it would take to fetch it from the content provider origin server.
Typically, a CDN is implemented as a combination of a content delivery infrastructure, a request-routing mechanism, and a distribution infrastructure. The content delivery infrastructure usually comprises a set of "surrogate" origin servers that are located at strategic locations (e.g., Internet network access points, Internet Points of Presence, and the like) for delivering copies of content to requesting end users. The request-routing mechanism allocates servers in the content delivery infrastructure to requesting clients in a way that, for web content delivery, minimizes a given client's response time and, for streaming media delivery, provides for the highest quality. The distribution infrastructure consists of on-demand or push-based mechanisms that move content from the origin server to the surrogates. An effective CDN serves frequently-accessed content from a surrogate that is optimal for a given requesting client. In a typical CDN, a single service provider operates the request-routers, the surrogates, and the content distributors. In addition, that service provider establishes business relationships with content publishers and acts on behalf of their origin server sites to provide a distributed delivery system. A well-known commercial CDN service that provides web content and media streaming is provided by Akamai Technologies, Inc. of Cambridge, Mass.
While content delivery networks provide significant advantages over prior techniques (e.g., proxy caching and mirroring), their performance is dependent generally on the performance of the Internet, which is far from an ideal communications medium. When a path in the network is either congested or faulty, packets sent over that path can get dropped or delayed. In spite of the existence of less congested paths, Internet routers might not be aware, willing or allowed to use these paths. Economical issues can also interfere with the best routing. Some routers might be configured to route packets using the cheapest route as opposed to the fastest route. Similarly, contractual issues also affect routing: for instance, first tier network providers must talk directly at their peering points and are not allowed to route through an intermediary provider. The peering points themselves often become a bottleneck at certain times of the day.
When a problem occurs on the Internet, it takes a certain amount of time for the routers to converge on a new view of the network, and to decide on the appropriate routing. Possible causes of these problems include router misconfiguration, software and hardware problems, and even fiber cuts. For example, a number of router problems can last about 5 minutes, and then simply go away. An actual misconfiguration can last significantly longer. In December, 2000, a router misconfiguration on a large network caused a large number of other routers to route their traffic to that particular router, thus overloading it. This condition lasted for a few hours, and it caused a significant slowdown on the Internet. A common problem in certain routers can cause partial network unreachability that can last for a few hours and can be very hard to detect. In June, 2001, two network providers stopped peering for administrative reasons. This made hosts in one provider's network simply unreachable by hosts in the other provider's network. This situation lasted for a few days.
In reality, it is not practical to understand all possible Internet problems and reasons why the best route between two points is not always followed. Edge servers operating within a content delivery network often have need to return to a content provider's origin server, e.g., when requested data is not available at the server or is otherwise stale, to obtain non-cacheable content, and the like. While sophisticated CDNs can route around network congestion and other problems, there remains a need to provide improved techniques to optimize the edge server-origin server communication path.
The present invention, in particular, addresses the problem of finding the best way to route data between content delivery network (CDN) regions and content providers, while quickly adjusting to changing network conditions.
It is a general object of the present invention to improve the speed and reliability of data transfers in a distributed networking environment, such as a content delivery network.
It is another general object of this invention to provide techniques for transferring data packets from one node to another node via an intermediate node based on network performance data collected over time.
It is another more specific object of the invention to provide a routing service within a distributed network such as a content delivery network whereby traffic is moved from a first region to a second region by sending it (i.e., "tunneling") through an intermediate region.
It is still another specific object to provide a novel routing mechanism, service or system in a content delivery network. The invention enables an edge server operating within a given CDN region to retrieve content (cacheable, non-cacheable and the like) more efficiently by selectively routing through the CDN's own nodes, thereby avoiding network congestion and hot spots. The invention thus enables an edge server to fetch content from an origin server through an intermediate CDN server or, more generally, enables an edge server within a given first region to fetch content from the origin server through an intermediate CDN region. As used herein, this routing through an intermediate server, node or region is sometimes referred to as "tunneling."
It is yet another more specific object of the invention to provide a routing service that predicts a best path for a data transfer between a source location (e.g., a content provider origin server) and a target location (e.g., a CDN edge server) by analyzing some performance metric common to a set of possible routes. In an illustrative embodiment, the performance metric is download time. In this embodiment, the performance metric is evaluated by having the edge server initiate a file download "race" in response to receiving a request for given content. In particular, a number of simultaneous downloads of the given content are initiated from the source location over a plurality of routes, some of which may include intermediate nodes. The winning path is then used for transfers between the source and the target locations for a given time period (e.g., until the next race).
According to an embodiment of the present invention implemented within a CDN, the identification of the intermediate nodes (and, thus, the alternative routes) to use for the race is determined in an off-line mapping process by performing given network traffic tests. In particular, a map making process operable within the CDN preferably determines the distances between given CDN "core" regions and the content provider, preferably based on ping data (age, loss and latency). The map making process then computes the best one and two-hop paths to the content provider from every CDN datacenter and publishes those paths (e.g., via DNS) as a "map." The usable intermediate CDN servers are typically set on a per content provider basis according to a given content provider policy, which is sometimes referred to as a strategy.
Thus, according to one embodiment of the invention, a map making process (which may be global-based) typically performs given network tests (e.g., pings) and uses the results of those tests to generate a map comprising a plurality of routes: the best route to the customer site, the best intermediate or "middle" region for tunneling, and the next best middle region. These routes may then be ordered according to some performance metric (e.g., actual download times) determined by periodic analysis (e.g., file download races) carried out locally by or under control of a given edge server.
According to another aspect of the present invention, a routing service may operate in one of two different modes: performance and failover. The performance mode continually polls the potential retrieval routes to rank their performance and uses the best route to retrieve content. The failover mode instructs the CDN edge server to go forward to the origin directly, but, in the event the direct route fails, to try one of the alternate routes. In the event the direct route fails, the alternate routes are tried in turn. This strategy need not use races to choose a route, because the direct route is to be used unless it completely fails to respond.
The foregoing has outlined some of the more pertinent features of the present invention. These features should be construed to be merely illustrative. Many other beneficial results can be attained by applying the disclosed invention in a different manner or by modifying the invention as will be described.
As described above, it is known in the art to delivery HTTP, streaming media and applications over an Internet content delivery network (CDN or ICDN). The present invention may leverage Internet CDN architecture and functionality such as now generally described.
As seen in
In one type of service offering, known as Akamai FreeFlow, from Akamai Technologies, Inc. of Cambridge, Mass., content is tagged for delivery from the CDN using a content migrator or rewrite tool 106 operated, for example, at a participating content provider server. Tool 106 rewrites embedded object URLs to point to the CDNSP domain. A request for tagged content is resolved through a CDNSP-managed DNS to identify a "best" region, and then to identify an edge server within the region that is not overloaded and that is likely to host the requested content.
Instead of using content provider-side migration (e.g., using the tool 106), a participating content provider may simply direct the CDNSP to serve an entire domain (or subdomain) by a DNS directive (e.g., a CNAME). In such case, the CDNSP may provide object-specific metadata to the CDN content servers to determine how the CDN content servers will handle a request for an object being served by the CDN. Metadata, as used herein, thus refers to the set of all control options and parameters for the object (e.g., coherence information, origin server identity information, load balancing information, customer code, other control codes, etc.), and such information may be provided to the CDN content servers via a configuration file, in HTTP headers, or in other ways. A configuration file is advantageous as it enables a change in the metadata to apply to an entire domain, to any set of directories, or to any set of file extensions. In one approach, the CDNSP operates a metadata transmission system 116 comprising a set of one or more servers to enable metadata to be provided to the CDNSP content servers. The system 116 may comprise at least one control server 118, and one or more staging servers 120 a-n, each of which is typically an HTTP server (e.g., Apache). Metadata is provided to the control server 118 by the CDNSP or the content provider (e.g., using a secure extranet application) and periodically delivered to the staging servers 120 a-n. The staging servers deliver the metadata to the CDN content servers as necessary.
Generalizing, the present invention may be implemented as a routing mechanism, service or system in a distributed networking environment. One preferred environment is a content delivery network wherein the present invention provides improved connectivity back to an origin server for HTTP traffic. As noted above, a CDN service provider typically segments its servers into regions, with each region comprising a set of content servers (e.g., up to about ten (10) servers) that preferably operate in a peer-to-peer manner and share data across a common backbone such as a local area network (LAN). The inventive routing technique enables an edge server operating within a given CDN region to retrieve content (cacheable, non-cacheable and the like) more efficiently by selectively routing through the CDN's own nodes, thereby avoiding network congestion and hot spots. The invention thus enables an edge server to fetch content from an origin server through an intermediate CDN server or, more generally, enables an edge server within a given first region to fetch content from the origin server through an intermediate CDN region. As used herein, this routing through an intermediate server, node or region is sometimes referred to as "tunneling."
The present invention identifies alternate paths from a CDN edge server to an origin server and uses these alternates to either improve the performance of content delivery, or to provide for fail-over in the event that the direct route is congested or otherwise unavailable. When an edge server contacts the origin server, the "direct" route typically is the route obtained through the Border Gateway Protocol (BGP). When the inventive routing technique is used, in contrast, alternate routes to the origin server are accessed by sending the request from the edge server to another intermediate (preferably CDN-managed) server/region before going to the origin. While one might assume that adding the intermediate step would reduce performance, it frequently improves performance, because CDN servers and regions typically are well-connected, and the indirect route can often bypass network congestion.
Preferably, given CDN edge servers are configured with a plurality (e.g., at least two) alternate routes to use in addition to the direct route to the origin. These intermediate servers preferably are specific to the given edge server and origin server to be contacted.
The routing system of the present invention may operate in one of two basic modes. In a performance mode, illustrated in
Preferably, the intermediate servers that are available for alternative routes between a given edge server (or region) and a given origin server are updated frequently (e.g., every 15-30 minutes) based on the current performance of connections between CDN servers.
In the preferred embodiment of the invention, the choice of which route an edge server should use to reach a given origin server preferably is made in three phases, which are described below: (a) a map of potential alternate routes is created (preferably by a global-based process called a mapmaker); (b) current ping data is used to identify the two best alternates for each CDN edge server (again, preferably by the mapmaker); and (c) each CDN edge server chooses among the direct route and the alternate routes based on the current actual performance with real requests for content. In particular, in a preferred embodiment, the CDN edge servers use occasional "races" among identical requests to determine which of the plurality (e.g., typically three (3)) possible routes is performing the best and then choose the route to use when going forward for the current and future requests. As will be described below, the parameters for these races are highly configurable to optimize the performance for the type of content being served and the intended result.
Before the inventive routing service is enabled for a given site, a large number (e.g., up to 100 CDN server regions) are identified as potential intermediate servers. These regions may be chosen on any available network performance metrics, e.g., such as analysis of ping data between CDN edge servers and well-connected CDN core servers and from these core servers to the origin. Once intermediate servers have been identified, they are used to define a "map" from CDN edge servers and the origin server. The invention preferably uses a custom map for each site because sites are hosted all over the Internet, on different backbones and in different geographic locations. As a result, the indirect routes that result in the best behavior are different for each CDN customer that uses the routing service. The indirect routes are then ordered such that the intermediate servers that provide the best performance for each given edge server are listed first. Notice that in the preferred embodiment the indirect routes are specific to each edge server and each customer origin server. The mapping is updated frequently (e.g., approximately every 15-30 minutes) based on current ping data to ensure optimal performance. When a CDN edge server requests its current map (preferably via a DNS lookup on a given routing service domain), it receives a given number (e.g., the two) top performing routes as alternates. The edge server then chooses among these alternate routes and the direct route by occasionally testing the performance of each connection, e.g., with a request for content.
As noted above, the inventive routing service preferably determines which route is the best at any given time by performing "races." These races consist of having the edge server forward a client request through the a given number (e.g., three (3)) available routes simultaneously and observing which route performed the best in serving the request. The information is then used to choose the primary route for future requests. Moreover, that route preferably remains the primary route until the next race is conducted. These races allow the routing service to react to changes in the Internet very quickly. A race preferably is initiated upon arrival of a request from an end user, and the CDNSP preferably configures how often a race will occur by setting one or more of the following preferred parameters: (a) the minimum amount of time that should elapse between races (with a default to, e.g., 30 seconds); (b) the minimum number of client requests that should arrive before each new race (with a default to, e.g., 40 requests); (c) the maximum amount of time that should elapse between races (with a defaults to, e.g., five minutes). The last parameter ensures that a race will occur at least this often and thus ensures that the best route is the one being used.
According to the invention, races are configurable to use either the actual object requested by a client (called a Request Object) or a predetermined test object on the origin server (called a Test Object), or both. Request Object races are generally preferable to Test Object races because the end user client that initiated the Request Object race can take advantage of the results, in particular, because the end user is served from the route that provided the fastest first-byte turn-around time. However, there are some cases where Request Objects are not possible and Test Objects are a good alternative. A Request Object race uses the object requested by the end user as the object it requests from the origin server. That is, if a browser requests http://us.a1.yimg.com/us.yimg.com/i/ww/m5v5.gif, the CDN edge server will request http://us.yimg.com/i/ww/m5v5.gif from all three forward servers. When this type of race is used, one may also configure a maximum number of bytes that should be downloaded through the losing routes. For example, the default configuration is to download up to 56K of data on the losing routes and then terminate the connection(s). The entire file is downloaded from the winning route, because this response is served to the end user client. The race preferably is scored based on the performance retrieving the 56 Kbytes of data, although this is not a limitation of the invention.
Races that use Request Objects may cause a problem for some sites because the origin server will receive three identical requests. If these requests impact billing or initiate some backend transaction, several anomalies many occur. A Test Object race uses an object specified in metadata as the object requested from the origin server across the three routes. The benefit of this is that the Test Object can be a special object that does not impact the origin server's logging or initiate backend transactions.
The edge server scores the races preferably based on three measurements: Request End Time; Turnaround Time (the first byte returned); Turnaround Time (the first byte returned). One can apply weightings to these measurements to influence the final scoring. For example, one could weight the transfer time more heavily to ensure that a rapid first byte score from a route would not erroneously determine the choice of the optimal route for fairly large files. Once the individual races have been scored, the CDN edge server compares those scores to choose the best route. Just as it is possible to weight one of the measurements in determining the score for the routes, it is also possible to favor the direct or indirect routes depending on the desired result. Normally the direct route is favored over the indirect routes. One could specify that, to be chosen, an indirect route must be faster than the direct route: by a given percent; by a specific number of milliseconds; by a given percent and an absolute number of milliseconds, or some combination thereof.
As noted above, the routing service operates in one of two modes: performance and failover. The performance mode continually polls the potential retrieval routes to rank their performance and uses the best route to retrieve content. The strategy preferably comes in two versions: origin-weighted and indirect-weighted. The origin-weighted strategy weights going direct to the origin as slightly more favorable than taking the indirect route. The indirect-weighted strategy favors the indirect route over the direct route to the origin server. This can be useful in providing a buffer for the origin server, while still allowing for direct contact to the origin in the event that the parent server is not providing sufficiently better service. The failover mode instructs the CDN edge server to go forward to the origin directly, but, in the event the direct route fails, to try one of the alternate routes. In the event the direct route fails, the alternate routes are tried in turn. This strategy need not use races to choose a route, because the direct route is to be used unless it completely fails to respond. The alternate routes preferably are determined based on the ping data and are listed in the order that they should be used.
Normally the CDN edge server will try to reach the origin server a given number of times with a specified timeout for each attempt. This timeout may be generally fairly long (several seconds), to ensure that a congested route does not cause the timeout to be triggered unnecessarily. When the inventive technique is enabled, each attempt to reach the origin server for a given request may involve trying the three possible routes in sequence as the connection attempts timeout. Because there are three routes to try, the timeout for each attempt can be relatively short (one second) in the hope that the timeout is a problem with the route, and not with the origin server. Only after all three routes have failed does the CDN edge server then attempt to reach the origin directly with the normal full-timeout. The logic being that if all routes have timed out, the problem is not likely to be a congested route, and it may be necessary to give the origin server more time to respond to the connection attempt. If that final connection attempt times out, the CDN edge server can initiate the fail-action to serve an error, serve default content, or retrieve content from an alternate origin server.
Referring to
Preferably, the datacenters that are usable as intermediate regions are set on a per strategy basis. Candidate regions may be removed for various reasons, e.g., the region is disabled, the region is at or near a given load capacity, or the like, and it may be desirable to allocate given intermediate regions to given customers.
The mapmaker constantly gathers liveness as well as ping data about the CDN regions. For example, a given agent in the CDN pings at least one edge server per region, and one IP address per each datacenter listed in the strategies. The mapmaker relies on the ability to predict how good a path is based on ping data. Ping latency and ping loss between pairs of regions may be used to compute value that is considered the effective distance between those two regions, or some other convenient estimate may be used. It is assumed that pings are symmetric with respect to pinger and pingee as illustrated in
For each content provider the mapmaker logically generates a number of maps, which can be identified as follows: Closest CP Map: the domain name of the closest mirror site for the content provider; Closest Reverse Proxy Map: the domain name of the closest reverse proxy if one exists; and Best Two-Hop Map: a list of usually two parent regions that should used to reach the mirror site. The first two maps are very similar in nature—they map each region to the closest possible source of data for each content provider. The Closest Reverse Proxy Map is not needed if there is no reverse proxy. Other factors can affect the choice of paths for the Two-Hop map. In general terms, the Two-Hop map encodes the two (or more) shortest paths from each region to mirror site of the content-provider, that go through at most one (active) middle region. The regions that can be used as an intermediate hop in the paths are usually the same regions that do the pinging, but the intermediate hop can also be explicitly specified as a subset of those regions.
As noted above, preferably the mapmaker relies on ping data as opposed to information about the actual download times each region experiences, or even what paths are actually used by the region to perform downloads. That information preferably is kept locally at each region. The guide process is the component within the edge server responsible for deciding which of the possible paths should be used. An edge server that is provisioned for the routing service thus includes the guide process, which is typically software. More generally, the guide computes an ordering or list of the possible paths/parents that could be used to obtain the content. The edge server tries parents in order: if a parent fails or times-out then the edge server will request content from the next parent. If the routing service is enabled for a given site and the edge server receives requests for data in that site, the edge server will occasionally, while responding to the request, measure the performance of a given number (e.g., three) simultaneous downloads from that site: one directly from the content provider, and a plurality (e.g., two) from the intermediate regions suggested by the two-hop map published by the mapmaker. The edge server then uses the results of this test to estimate which route should be used to access that site, until a new test occurs. If a long time elapses without a test, the previous results expire. It may also be desirable to allow for the downloads between tests to affect the predicted quality of each of the routes, thus allowing for an even faster response time to a network problem, without requiring additional tests.
When a request comes in and is not purely cacheable, it results in the download of a file from the content provider. This is what is sometimes called a forward request On forward requests for sites that are provisioned to use the routing system and service, the results of the previous test are used to choose the route for the download. The exceptions to this rule occur if the content can be and is itself used for a test or if there are no recent test results available. In the first case, the download preferably occurs through all (e.g., three (3)) routes, and the one that returns the first byte faster will be used to serve a client. The other two will have their results recorded and will be allowed to proceed for up to 56 k bytes, after which they will be aborted. If however, the content cannot be used for a test and there are no recent test results available, a default ranking is used to decide on which route to use. The default order can either specify that the direct or the indirect route should be favored.
Preferably, tests are linked to a forward request, meaning they do not happen if there has been no forward request to that site. In the case of a forward request, a test is by default preferably run when: no test occurred in a given number of minutes, or at least a given number of seconds have passed since the last test, or if at least a given number of forward requests from this site did not result in tests. These settings try to ensure that for a site with a reasonable amount of volume there will always be fresh statistics for the download paths while at the same time limiting the amount of testing that is performed over the network as a whole.
Once a decision is made to use the routing service, one must also decide on what map to use. Preferably, the CDNSP creates a new strategy for the customer's site. Once the mapmaker machines receive the new strategy, they update their list of pingees, start collecting ping data for the updated list, and start producing maps for the additional strategy. Once a map is being published for the site, the routing service is enabled for the site. When an edge server receives a request that results in a forward request, the edge server has to decide what route to take.
As noted above, preferably the CDN includes a metadata transmission system. Metadata can be used to control the routing service, as is now described. Some representative metadata tags that may be used are listed below, with a brief explanation of their meanings:
Metadata Tag
the map that should be used
url to be used in tests
test-objects should be used, i.e., tests
should use the selected url
desired time between tests
ratio between direct and indirect test
results before indirect is used
time before routing fails-over to
another parent if it cannot connect
specifies if indirect should be favored
in the absence of tests results
Thus, to provide an illustrative embodiment, the use-hierarchy tag is first enabled to enable the routing service to take effect, e.g., with default settings. In addition to the use-hierarchy tag, the following metadata tags (among others) could then be set to configure the routing service for the best performance:
As noted above, the inventive routing service can be configured in a failover mode that does not use performance testing. In such case, the edge server is given three parents: that of the content provider (direct) and two indirect routes. In failover mode, the edge server contacts direct first. If for some reason this connection fails, the edge server will try its first parent, then its second parent, and finally back to direct. Because the mapmaker is constantly publishing up-to-date indirect routes, these failover servers are likely to serve the content successfully. When failover mode is used, the configuration may include a metadata tag for disabling test clients:
<md name="cdnroute-no-test-client">on</md>.
The following provides additional detail regarding how to configure races in the performance mode. Preferably, races for a given route strategy occur when the edge server receives a request for content configured to use that strategy. When a request arrives, the server preferably looks at two pieces of data: the amount of time since the last race for this strategy, and the number of connections since the last race for this strategy. There are preferably three (3) conditions that can be applied (e.g., via metadata) to these two pieces of information:
cdnroute-max-time-before-tc: The um amount of time before performing a race (default: 5 m): <md name="cdnroute-max-time-before-tc">500</md>
cdnroute-max-req-before-tc: The maximum number of requests after which a race will be performed (default: 40): <md name="cdnroute-max-req-before-tc">1000</md>
cdnroute-min-tc-period: The minimum amount of time that must have elapsed before performing a race (default: 30 s): <md name="cdnroute-min-tc-period">120</md>.
Request Object races use the client request for the test client and use the settings above to control the frequency of the races. In addition to these settings, there may be additional tags for controlling the use of request object races. For example, a tag such as racer-max-byte takes as its value a number in bytes from 0 to 2 MB. It defaults to 56K and may be set as follows: <md name="racer-max-byte">32000</md>. This metadata limits the amount of data downloaded by the losing routes when a race is performed. The following header will cause the inclusion of a header binding the test client requests together so that the duplicate requests can be identified as such: -CDNSP-TC-Identifier: t[co]-<IP>-<edgeserver_curtime>-<incrementing counter>. The initial string is "tc" for test clients and "to" for test objects. Test objects are like request objects, except the URL being requested is replaced with a fixed URL configured through metadata. The following is an example of how to set this up:
An illustrative technique for scoring the races is now described. This description is merely representative. The score for each of the routes is made up of the request_end_time, the turnaround_time and the transfer_time. These times are multiplied by their specified weights and then summed. Each of the items may be weighed using a separate metadata tag and, preferably, each of these tags has an equivalent value. In this example, the cdnroute weighting tags all take as their value a number between −20 and +20. The default setting is one "1".
The cdn route-direct-scale tag is used to apply a weighting to the total score for the direct route. This value of the tag is a number, and it works like a percent. The scores of the indirect parents are automatically multiplied by 100, so a number less than 100 will decrease the score for the direct route relative to the other routes and make it more likely to be chosen. For example, setting the value to 50 will favor the direct route twice as much as the indirect route. A setting of 120 would favor the indirect routes. The tag may be configured as follows: md name="cdnroute-direct-scale">100</md>. The cdnroute-direct-scale takes as its value a number between 0 and 255. The default value is 100, but 95 is a useful value.
The cdnroute-absolute-direct-scale tag takes a number from −2000 to +2000 that represents a number of milliseconds. This is normally set to 10. The number is an offset that is subtracted from the direct route score to ensure that the indirect route must be at least this amount better than the direct route before it is chosen as the best route.
One of ordinary skill in the art will appreciate that the technique of tunneling through an intermediate CDN node for the purpose of faster and more reliable region-to-origin server may be extended to facilitate region-to-region communication. Each CDN server for a given client request may play one of three different roles: edge-server, middle-server or root-server. An edge-server is a server receiving an HTTP request from a client. A root-server is a CDN server that retrieves the content directly from the content provider origin server. Thus, a root-server may be co-located or located near a content provider origin server, although this is not a requirement. Any CDN server node that lies between the edge-server and the root-server is a middle-server. These CDN servers simply forward packets between the edge and root servers, i.e., a child CDN server and its parent CDN server.
Consistent with the methodology described above, the following are representative steps that would take place after an edge-server gets an HTTP request from a client. First, the edge-server determine a list of parent IP addresses as well as an IP address for the content provider origin server. In an illustrative embodiment, such information is obtained from a map generated by a mapmaker machine. To obtain the map, the edge server preferably issues a name query (i.e., a DNS lookup). Any convenient domain name construct can be used to direct the requesting server to a given map for a given CDN customer. Then, the edge server determines if it is to apply a race and/or Rsync to the request. If a race is used, the client becomes a VIP-client. Based on how long it has been since the edge server has picked a VIP-client, the edge server determines if this client will be a VIP-client or not. As noted above, certain rules or policies may be used to determine whether the edge server picks the client to be a VIP-client. Thus, for example, the request may be considered to be from a VIP-client according to the following constraints: (a) if a VIP-client has been picked for this content provider in the last 30 seconds, do not pick this client; (b) if no VIP-client has been picked for this content provider in the last 5 minutes, pick this client; (c) if no VIP-client has been picked for this content provider in the last 40 downloads, pick this client; (d) for large downloads ensure that the network is not overloaded by aborting a download after the first 56 Kbytes are received over a particular route. Of course, the above rules are merely exemplary.
It is assumed that parent zero is a root-server. The edge-server preferably keeps a running average of download times per content provider and parent over the VIP-client downloads. If the client is a VIP-client, the edge-server will download a copy of the file from all or some number of the parents. It preferably returns the first file to arrive to the client and records the time for each of the downloads. This information is then used in the running averages. If the client is a normal client, then the edge-server picks the parent with the best estimator that is computed from the running averages. If the content provider is multi-homed, it has servers in more than one location. In such case, the edge-server will use a default method provided by the content provider to determine the closest home.
As described above, a Closest Reverse Proxy Map may be used where the CDN service provider maintains reverse proxy regions. This map may be generated as follows. Let Ŕ be the list of RootProxy regions for some content provider. For each region R, consider all two-hop paths from the region R to some region in Ŕ. Return the shortest two such paths. Not all two-hop paths are considered. Let D be the distance from R to the closest RootProxy. Discard a path R? R1? R2, if one of the following is true: (a) 0.8×D<Dist(R1, R2) (b) 1.2×D<Dist(R, R1)+Dist(R1, R2).
The following algorithm may also be used to estimate the distance between regions. To adjust ping time and ping loss into a distance, use the following Dist=(100/(101-%Loss)2×pingTime. This formula has the following effect: for a 10% loss, increase the ping time by 20%; for a 20% loss, increase the ping time by 50%; for a 30% loss, increase the ping time by 100%. If desired the ratio above may be cubed instead of squared.
The tunneling technique provides a significant (e.g., two times) decrease in average file download time. Although not required, preferably the tunneling technique is coupled with a data compression and difference algorithm so that the number of bytes transmitted from a root-server to an edge-server is also reduced. A representative algorithm for this purpose is Rsync, although other such algorithms may be used. The use of Rsync (or a similar approach wherein file differences are transmitted) decreases bandwidth used on expensive longhaul links and provides addition reduction in download time due to the smaller file sizes being transmitted.
The present invention provides numerous advantages. The inventive techniques are especially useful for any CDN customer seeking to optimize performance in content delivery and for many customers interested in improved reliability. The performance benefit is greatest when the connection from an edge server to the customer's origin is frequent, because this is the transfer that is optimized by the present invention. This applies regardless of whether the connection is used to transfer an entire file or an If-Modified-Since request to revalidate content or authorize a request before serving the client. Specifically, any content that uses dynamic content assembly, no-store, bypass-cache, centralized authorization, zero-TTL or low TTL settings, is a good candidate for use of the techniques described above. Of course, the above is merely illustrative and should not be taken to limit the scope of the present invention in any way.
One of ordinary skill in the art will appreciate that the inventive technique for tunneling data from the content provider origin server (or a reverse proxy) to the edge server is not meant to be limited to data transfers in one direction. There are many instances where data transfer will go from the edge server to the origin, e.g., when an application executing on the edge server provides data to the origin. Thus, the inventive technique should be considered bi-directional or uni-directional.
As described above, the present invention is not limited to use of file download races to prioritize the set of routes that are exposed in a given content provider map. Other performance metric tests may be used in addition to or in lieu of the file downloads. For example, a simple liveness check may tell the edge server that a given route identified in the map is no longer active. Moreover, other well-known techniques may be used to determine which path (either direct or indirect) is the best or optimal one for edge server to origin/proxy server (or vice versa) communications. Such additional methodologies may be used to select the optimal routes include, without limitation: analysis of BGP data, analysis of historical TCP/IP statistics, measurements conducted as a result of exchanges of data between regions, additional network statistics generated from traceroutes, pings, udp traffic, and combinations of the above.
Having described our invention, what we now claim is set forth below:
|Brevet cité
|Date de dépôt
|Date de publication
|31 déc. 1980
|17 aoűt 1982
|Bell Telephone Laboratories, Incorporated
|Dynamic, non-hierarchical arrangement for routing traffic
|18 oct. 1996
|14 déc. 1999
|Intervu, Inc.
|System and method for optimized storage and retrieval of data on a distributed computer network
|19 mai 1999
|22 aoűt 2000
|Massachusetts Institute Of Technology
|Global hosting system
|31 mars 1998
|19 sept. 2000
|Siemens Information And Communication Networks, Inc.
|System for providing enhanced security for transactions transmitted through a distributed network
|18 juin 1999
|14 aoűt 2001
|Digital Island, Inc.
|On-demand overlay routing for computer-based communication networks
|3 juil. 2000
|31 aoűt 2004
|Fastforward Networks
|Content distribution system for operation over an internetwork including content peering arrangements
|31 aoűt 2000
|21 sept. 2004
|Neoris Logistics, Inc.
|Centralized system and method for optimally routing and tracking articles
|21 sept. 2000
|7 mars 2006
|Akamai Technologies, Inc.
|Internet content delivery service with third party cache interface support
|Anderson et al., Resilient Overlay Networks, 18th ACM Symp. on Operating Systems Principles (SOSP), Oct. 2001.
|Cranor et al., Enhanced Streaming Services in a Content Distribution Network, IEEE Internet Computing, Aug. 2001.
|Johnson et al., The Measured Performance of Content Distribution Networks, May 2000.
|Kirshnamurthy et al., On the Use and Performance of Content Distribution Networks, AT&T Labs, Aug. 2001.
|Lazar et al., Exploring Content Delivery Networking, IT Pro, Aug. 2001.
|Savage et al., Detour: Informed Internet Routing and Transport, IEEE Micro, Jan.-Feb. 1999.
|Wessels et al., Application of Internet Cache Protocol (ICP), Version 2, Internet Request for Comment (RFC) 2187, Sep. 1997.
|Wessels et al., Internet Cache Protocol (ICP), Version 2, Internet Request for Comment (RFC) 2186, Sep. 1997.
|Wessels, Release Notes for version 1.1 of the Squid cache, Apr. 1997.
|Brevet citant
|Date de dépôt
|Date de publication
|6 nov. 2006
|20 mai 2008
|Akamai Technologies, Inc.
|Method and system for tiered distribution in a content delivery network
|13 nov. 2006
|20 mai 2008
|Akamai Technologies, Inc.
|Method and system for providing on-demand content delivery for an origin server
|27 oct. 2004
|8 déc. 2009
|Cisco Technology, Inc.
|Method and apparatus for forwarding data in a data communications network
|26 janv. 2005
|9 mars 2010
|I2 Telecom Ip Holdings, Inc.
|System and method of binding a client to a server
|22 juil. 2005
|6 avr. 2010
|Cisco Technology, Inc.
|Method and apparatus for advertising repair capability
|25 sept. 2006
|20 avr. 2010
|Cisco Technology, Inc.
|Forwarding data in a data communications network
|1 aoűt 2005
|27 avr. 2010
|Limelight Networks, Inc.
|Heavy load packet-switched routing
|9 janv. 2003
|27 avr. 2010
|Cisco Technology, Inc.
|Method and apparatus for constructing a backup route in a data communications network
|3 mars 2004
|4 mai 2010
|Cisco Technology, Inc.
|Method and apparatus for computing routing information for a data communications network
|2 juin 2008
|24 aoűt 2010
|International Business Machines Corporation
|Method and apparatus for scheduling grid jobs
|31 mai 2007
|19 oct. 2010
|International Business Machines Corporation
|Optimization process and system for non-multiplexed peer-to-peer architecture
|24 oct. 2005
|9 nov. 2010
|International Business Machines Corporation
|Method and apparatus for presenting a visualization of processor capacity and network availability based on a grid computing system simulation
|31 mai 2007
|30 nov. 2010
|International Business Machines Corporation
|Coalition formation and service provisioning of bandwidth sharing AD HOC networks
|5 juil. 2005
|7 déc. 2010
|Cisco Technology, Inc.
|Method and apparatus for constructing a repair path for multicast data
|1 juin 2004
|7 déc. 2010
|Cisco Technology, Inc.
|Method and apparatus for forwarding data in a data communications network
|24 oct. 2005
|14 déc. 2010
|International Business Machines Corporation
|Method and apparatus for scheduling grid jobs
|31 mai 2007
|28 déc. 2010
|International Business Machines Corporation
|Optimization process and system for multiplexed gateway architecture
|15 juil. 2003
|4 janv. 2011
|Cisco Technology, Inc.
|Method and apparatus for forwarding a tunneled packet in a data communications network
|15 janv. 2003
|11 janv. 2011
|Cisco Technology, Inc.
|Method and apparatus for determining a data communication network repair strategy
|31 mai 2007
|18 janv. 2011
|International Business Machines Corporation
|Systems and methods for establishing gateway bandwidth sharing ad-hoc networks
|31 mai 2007
|1 mars 2011
|International Business Machines Corporation
|Efficiency and resiliency enhancements for transition states in ad hoc networks
|20 sept. 2007
|19 avr. 2011
|Akamai Technologies, Inc.
|Optimal route selection in a content delivery network
|22 févr. 2005
|26 avr. 2011
|Cisco Technology, Inc.
|Method and apparatus for constructing a repair path around a non-available component in a data communications network
|13 juin 2007
|10 mai 2011
|Cisco Technology, Inc.
|Fast re-routing in distance vector routing protocol networks
|31 juil. 2006
|14 juin 2011
|Limelight Networks, Inc.
|Routing under heavy loading
|31 mai 2007
|12 juil. 2011
|International Business Machines Corporation
|Payment transfer strategies for bandwidth sharing in ad hoc networks
|13 sept. 2005
|9 aoűt 2011
|International Business Machines Corporation
|Grid network throttle and load collector
|31 mai 2007
|18 oct. 2011
|International Business Machines Corporation
|Demand pull and supply push communication methodologies
|19 mai 2008
|18 oct. 2011
|Akamai Technologies, Inc.
|Method and system for providing on-demand content delivery for an origin server
|10 déc. 2009
|13 déc. 2011
|Akamai Technologies, Inc.
|Load balancing using IPv6 mobility features
|10 juin 2008
|10 janv. 2012
|International Business Machines Corporation
|Grid project modeling, simulation, display, and scheduling
|17 oct. 2011
|8 mai 2012
|Akamai Technologies, Inc.
|Load balancing using IPV6 mobility features
|27 févr. 2010
|26 juin 2012
|Yottaa Inc
|System and method for computer cloud management
|14 nov. 2008
|10 juil. 2012
|At&T Intellectual Property I, Lp
|Interdomain network aware peer-to-peer protocol
|15 déc. 2008
|24 juil. 2012
|International Business Machines Corporation
|Opening document stored at multiple database replicas
|17 oct. 2011
|7 aoűt 2012
|Akamai Technologies, Inc.
|Origin server protection service apparatus
|31 mai 2007
|21 aoűt 2012
|International Business Machines Corporation
|System and method for fair-sharing in bandwidth sharing ad-hoc networks
|31 oct. 2003
|16 oct. 2012
|Aol Inc.
|Managing access to digital content sources
|31 mai 2007
|27 nov. 2012
|International Business Machines Corporation
|Formation and rearrangement of lender devices that perform multiplexing functions
|30 mars 2012
|25 déc. 2012
|Akamai Technologies, Inc.
|Server failover using IPV6 mobility features
|5 avr. 2012
|19 févr. 2013
|International Business Machines Corporation
|Opening document stored at multiple database replicas
|6 mai 2011
|16 avr. 2013
|Limelight Networks, Inc.
|Routing under heavy loading
|21 mai 2013
|Akamai Technologies, Inc.
|Site acceleration with content prefetching enabled through customer-specific configurations
|12 mars 2010
|20 aoűt 2013
|Limelight Networks, Inc.
|Publishing-point management for content delivery network
|31 mai 2007
|27 aoűt 2013
|International Business Machines Corporation
|Optimization process and system for a heterogeneous ad hoc Network
|7 juin 2012
|10 sept. 2013
|At&T Intellectual Property I, L.P.
|Interdomain network aware peer-to-peer protocol
|4 aoűt 2010
|24 sept. 2013
|Cisco Technology, Inc.
|System and method for providing a link-state path to a node in a network environment
|28 sept. 2010
|5 nov. 2013
|Amazon Technologies, Inc.
|Request routing management based on network components
|25 juin 2009
|5 nov. 2013
|Akamai Technologies, Inc.
|Generation and use of network maps based on race methods
|19 nov. 2007
|26 nov. 2013
|AlterWAN Inc.
|Wide area network with high quality of service
|13 déc. 2007
|10 déc. 2013
|Nbcuniversal Media, Llc
|Accelerating peer-to-peer content distribution
|31 aoűt 2009
|10 déc. 2013
|Hipcricket, Inc.
|System and method of binding a client to a server
|31 mai 2007
|31 déc. 2013
|International Business Machines Corporation
|Formation and rearrangement of ad hoc networks
|27 mars 2009
|1 avr. 2014
|Amazon Technologies, Inc.
|Dynamically translating resource identifiers for request routing using popularity information
|13 févr. 2013
|29 avr. 2014
|Amazon Technologies, Inc.
|Request routing based on class
|17 nov. 2008
|20 mai 2014
|Amazon Technologies, Inc.
|Request routing utilizing cost information
|11 mars 2013
|17 juin 2014
|Amazon Technologies, Inc.
|Content management
|27 mars 2009
|17 juin 2014
|Amazon Technologies, Inc.
|Request routing utilizing popularity information
|16 juin 2009
|15 juil. 2014
|Amazon Technologies, Inc.
|Managing resources using resource expiration data
|1 juin 2005
|26 aoűt 2014
|Akamai Technologies, Inc.
|Network traffic load balancing system using IPV6 mobility headers
|28 sept. 2010
|26 aoűt 2014
|Amazon Technologies, Inc.
|Request routing in a networked environment
|22 févr. 2012
|2 sept. 2014
|Level 3 Communications, Llc
|Content delivery network analytics management via edge stage collectors
|26 sept. 2011
|30 sept. 2014
|Limelight Networks, Inc.
|Dynamic route requests for multiple clouds
|22 juin 2005
|28 oct. 2014
|Core Wireless Licensing S.A.R.L.
|System and method for establishing peer to peer connections between PCS and smart phones using networks with obstacles
|28 sept. 2010
|30 déc. 2014
|Amazon Technologies, Inc.
|Latency measurement in resource requests
|28 sept. 2010
|6 janv. 2015
|Amazon Technologies, Inc.
|Latency measurement in resource requests
|21 mars 2009
|6 janv. 2015
|Level 3 Communications, Llc
|Handling long-tail content in a content delivery network (CDN)
|29 oct. 2013
|6 janv. 2015
|Amazon Technologies, Inc.
|Network resource identification
|28 sept. 2010
|20 janv. 2015
|Amazon Technologies, Inc.
|Request routing management based on network components
|26 aoűt 2013
|31 mars 2015
|Amazon Technologies, Inc.
|Translation of resource identifiers using popularity information upon client request
|8 juil. 2011
|7 avr. 2015
|Ming Li
|Content delivery prediction and feedback systems
|28 sept. 2010
|7 avr. 2015
|Amazon Technologies, Inc.
|Point of presence management in request routing
|29 avr. 2013
|7 avr. 2015
|Amazon Technologies, Inc.
|Request routing processing
|21 nov. 2011
|14 avr. 2015
|May Patents Ltd.
|Apparatus and method for using and solving linear programming problem and applications thereof
|6 mai 2013
|14 avr. 2015
|Amazon Technologies, Inc.
|Locality based content distribution
|20 janv. 2006
|21 avr. 2015
|Cisco Technology, Inc.
|Link grouping for route optimization
|14 mars 2013
|28 avr. 2015
|Amazon Technologies, Inc.
|Updating routing information based on client location
|17 mai 2013
|28 avr. 2015
|Amazon Technologies, Inc.
|Request routing using network computing components
|3 juin 2013
|28 avr. 2015
|Amazon Technologies, Inc.
|Request routing utilizing client location information
|17 mai 2013
|5 mai 2015
|Amazon Technologies, Inc.
|Content delivery reconciliation
|12 sept. 2012
|19 mai 2015
|International Business Machines Corporation
|Formation and rearrangement of ad hoc networks
|13 sept. 2012
|30 juin 2015
|Aol Inc.
|Managing access to digital content sources
|4 juin 2013
|14 juil. 2015
|Amazon Technologies, Inc.
|Translation of resource identifiers using popularity information upon client request
|20 juin 2012
|14 juil. 2015
|Amazon Technologies, Inc.
|Managing request routing information utilizing performance information
|22 mars 2013
|28 juil. 2015
|Limelight Networks, Inc.
|Routing under heavy loading
|27 aoűt 2012
|4 aoűt 2015
|International Business Machines Corporation
|Formation and rearrangement of lender devices that perform multiplexing functions
|4 nov. 2013
|11 aoűt 2015
|Amazon Technologies, Inc.
|Request routing management based on network components
|11 mars 2013
|8 sept. 2015
|Amazon Technologies, Inc.
|Managing secure content in a content delivery network
|20 sept. 2012
|15 sept. 2015
|Amazon Technologies, Inc.
|Automated profiling of resource usage
|30 juin 2003
|30 déc. 2004
|Hennessey Wade L.
|Method and apparatus for establishing peering rules for distributed content delivery
|31 oct. 2003
|24 févr. 2005
|Brown Scott K.
|Managing access to digital content sources
|14 oct. 2003
|14 avr. 2005
|Previdi Stefano Benedetto
|Method and apparatus for generating routing information in a data communication network
|11 mars 2003
|26 mai 2005
|Epstein Michael A.
|Using timing signals to determine proximity between two nodes
|13 févr. 2004
|8 sept. 2005
|Champagne Andrew F.
|Method and system for monitoring border gateway protocol (BGP) data in a distributed computer network
|1 juin 2004
|1 déc. 2005
|Previdi Stefano B
|Method and apparatus for forwarding data in a data communications network
|26 janv. 2005
|9 févr. 2006
|Cooney John M
|System and method of binding a client to a server
|27 oct. 2004
|27 avr. 2006
|Shand Ian Michael C
|Method and apparatus for forwarding data in a data communications network
|22 févr. 2005
|24 aoűt 2006
|Bryant Stewart F
|Method and apparatus for constructing a repair path around a non-available component in a data communications network
|22 juin 2005
|28 déc. 2006
|Nokia Corporation
|System and method for establishing peer to peer connections between PCS and smart phones using networks with obstacles
|5 juil. 2005
|25 janv. 2007
|Bryant Stewart F
|Method and apparatus for constructing a repair path for multicast data
|31 juil. 2006
|1 févr. 2007
|Raciborski Nathan F
|Routing Under Heavy Loading
|1 aoűt 2005
|1 févr. 2007
|Limelight Networks, Inc.
|Heavy load packet-switched routing
|9 janv. 2003
|15 févr. 2007
|Miles Kevin G
|Method and apparatus for constructing a backup route in a data communications network
|22 juil. 2005
|22 févr. 2007
|Previdi Stefano B
|Method and apparatus for advertising repair capability
|6 nov. 2006
|8 mars 2007
|Dilley John A
|Method and system for tiered distribution in a content delivery network
|13 sept. 2005
|15 mars 2007
|Viktors Berstis
|Method and apparatus for a grid network throttle and load collector
|31 aoűt 2005
|29 mars 2007
|Computer Associates Think, Inc.
|Deciding redistribution servers by hop count
|24 oct. 2005
|26 avr. 2007
|Viktors Berstis
|Method and apparatus for grid multidimensional scheduling viewer
|20 juin 2011
|19 avr. 2012
|Akamai Technologies, Inc.
|Extending a content delivery network (CDN) into a mobile or wireline network
|26 juil. 2012
|International Business Machines Corporation
|Opening Document Stored at Multiple Database Replicas
|23 aoűt 2012
|Level 3 Communications, Llc
|Analytics management
|8 juil. 2011
|10 janv. 2013
|Ming Li
|Content delivery prediction and feedback systems
|14 mars 2013
|Aol Inc.
|Managing access to digital content sources
|4 nov. 2013
|27 févr. 2014
|Amazon Technologies, Inc.
|Request routing management based on network components
|7 déc. 2012
|12 juin 2014
|Verizon Patent And Licensing Inc.
|Selecting a content delivery network
|Classification aux États-Unis
|370/227, 370/351, 370/228
|Classification internationale
|Classification coopérative
|Classification européenne
|1 mars 2002
Owner name: AKAMAI TECHNOLOGIES, INC., MASSACHUSETTS
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BORNSTEIN, CLAUDSON F.;CANFIELD, TIMOTHY K.;MILLER, GARYL.;AND OTHERS;REEL/FRAME:012671/0550;SIGNING DATES FROM 20020228 TO 20020301
|15 juin 2007
Owner name: AKAMAI TECHNOLOGIES, INC., MASSACHUSETTS
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SUNDARAM, RAVI;REEL/FRAME:019437/0529
Effective date: 20070614
|24 févr. 2011
|Fee payment
Year of fee payment: 4
|11 mars 2015
|Fee payment
Year of fee payment: 8


text:
Hong Kong ferry fee increases slammed after HK$190m subsidy
Government says it will closely scrutinise future applications
Most Popular
The government came under fire today for approving ferry fee increases – after operators were granted a HK$190 million subsidy last year – and admitted it could not rule out more raises in the future.
However, Yau Shing-mu, the undersecretary for transport and housing, told the Legislative Council transport panel meeting that any application for fee rises during the three-year term of their renewed licence would be tightly scrutinised.
"There is no guarantee that ferry companies would definitely not apply for fee rises, because the oil prices are hard to predict," said Yau. "But if such special occasions arise, the government will be the gatekeeper."
In March, the Transport Department approved plans by New World First Ferry Services and Hong Kong and Kowloon Ferry to increase the fare of their six ferry lines by about 5 per cent and 6 per cent due to rising oil prices and salary increases. The increase on Central-Mui Wo line took effect in April and the remaining lines will have the fees raised from July.
The approval came after a HK$190 million subsidy to the six routes during the three-year licence period from this year to 2017 passed the Legco's Finance Committee in July last year.
"I have never seen such outlandish things before," said panel member Kwok Ka-ki. "It's horrifying that you've used taxpayers' money and ended up being slapped back [by ferry companies]."
Kwok, representing the medical sector, said ferry services had not improved even after the subsidy and fee rises, citing poor hygiene in ferry piers, outdated facilities on ferries and heavily polluting fuel being used.
Other legislators agreed, adding that although many ferry staff's salaries had been increasing by more than 10 per cent each year, their starting salaries were very low.
Yau said the subsidy was set to cover part of the two ferry companies' operational costs of about HK$370 million and that the government had taken the subsidy into consideration when processing the companies' applications for a fee rise. He said the operators had been suffering from a low number of passengers which is showing sings of decreasing further.
The fee rise had been lower than the inflation rate, he said.
Legislators also asked about the progress on starting a new ferry route between Aberdeen to Cheung Chau, which was requested by local district councillors and residents.
Principal transport officer Albert Su Yau-on said the government had finished its study on the route and felt that it was technically practicable. He said the department was now waiting for district councillors to collect and submit local residents' opinions on potential services and prices before starting public bidding for an operator.
Su said the bidding process was expected to finish in four to six months.
This article is now closed to comments
(((Yau said the subsidy was set to cover part of the two ferry companies' operational costs of about HK$370 million and that the government had taken the subsidy into consideration when processing the companies' applications for a fee rise.)))
Shame on u ferry, to make abuse of the taxpayers! Cost-Subsidy = turnover/omzet... 370-190=180 turnover!!! And i know i pay a lot of travel cost for this ferry!!! How many consumers have travel in one day on this ferry? dose consumers x 1day x 1 travel bill/omzet x 365 day= turnover/omzet? Think,Think, Think Government! Second, the routine maintenance cost of year, maintenance inspection of the ferry are none, some ferry when i travel are very ugly? 3e, wage, salary of personnel how much? 1 ferry boat, i saw maybe only 7 people work there in 1 boat? 2 man open/ close the door, 1 captain, 1 driver, 1 cleaner and maybe 2 in cabinne? i don't know but i don't see a lot of workers there in the boat? how many cost this people wages? 7x ... x 1 day x 365d = cost? if u can calculation u know there is a lot of abuse of this money... so, no more high fares payment to the ferry boss? Please, don't spend the tax money in a ****, lost, abuse situation, u know as government, we know as consumers... freedom of speech...

subset: commoncrawl


text:
Whole Foods Market is a full service natural foods marketplace serving Las Vegas since 1998. We delight our customers with the highest quality natural and organic foods and premium body care available.Select from an array of organic fruits and veggies, fresh meat and seafood, and gourmet grocery items- we provide every needed ingredient for nutritious home-cooking. And because our busy lives don't always allow for shopping, chopping and baking, we offer those on the go delicious options including hot entrees, savory deli salads, fresh smoothies and scrumptious baked goods.
You may email your local store, but please remember that our team is working throughout the store (not at computers) and emails may not be answered immediately.
Please use the dropdown below to help us direct your question or comment appropriately. After choosing a topic you will be directed to our comment form.
Selecting a store allows you to see that store's content throughout the site, such as sales, store events, and more.
These links take you off wholefoodsmarket.com
California Transparency in Supply Chains Act Disclosure (PDF)
Copyright 2015 Whole Foods Market IP. L.P. | Terms of Use | Privacy Policy | Site Map | Site Information
We have sent a confirmation email to {* emailAddressData *}. Please check your email and click on the link to activate your account.
Please confirm the information below before signing in. Already have an account? Sign In.{* #registrationForm *} {* traditionalRegistration_firstName *} {* traditionalRegistration_lastName *} {* traditionalRegistration_emailAddress *} {* traditionalRegistration_password *} {* traditionalRegistration_passwordConfirm *} {* traditionalRegistration_displayName *} {* traditionalRegistration_postalCode *} {* captcha *} By clicking "Create Account", you confirm that you accept our terms of service and have read and understand privacy policy.
We've sent an email with instructions to create a new password. Your existing password has not been changed.


text:
Are you ready to energize your spirit? Looking to invigorate each day with a new sense of vitality and a positive attitude? Then come to St. Clair Beach Retirement Community, a Signature Retirement Living community devoted to helping you bring vitality to everything you do, with a comprehensive program of activities and services designed to help make every day a great day!
Welcome to St. Clair Beach Retirement Community. We are located on beautiful Riverside Drive where we have stunning water and park views. Come visit us and see how our residents live a vibrant lifestyle . Our dedicated and caring staff ensure our residents receive unmatched service and enviable level of choice whether it is in Dining Services or our wide-ranging activities and services, there is always something fun and entertaining do for everyone. Come find out why St. Clair Beach Retirement is the place to be.
Andrew Crow
Executive Director
Care Types Offered:
- Independent Living
- Personal Onsite Care
- Respite Care
- Vacation Stays
Photo Gallery
Visit Website Now


text:
News Releases
Gaming for Adults with Special Needs
Mondays, Nov 2, Dec 7
1:30-3:30pm • Carman Community Room
Please join us for drop in game time geared for adults with special needs. Wii games and several board games will be available. Feel free to bring your own game. Caregivers must remain in the room. No registration is required.


text:
|Key Info
PL2 is located in Plaka, the most desirable inner city address in Athens. The apartment has one large bedroom with a Queen Size bed and 100% linen sheets and quality pillows.
It has a remodelled marble kitchen that looks out to the courtyard and a bathroom with mosaic floors, bath and shower over and air condition.
PL2 has a patio with a dining table that can cater up to 4 people.
It is located on the semi-basement. During the winter time the boiler for the heater is sometimes audible.
It is three minutes away from Syntagma Metro Station and five minutes' walk from Acropolis metro station.
Within minutes walk you enter the pedestrian walkway that allows you to amble in the historic city centre, visit the Acropolis, the Old Olympic Stadium or walk in picturesque Plaka.
PL2 is in the very middle of Athens. There are numerous restaurants, tavernas and bars to cater all tastes. You will also find organic food markets, super markets and fruit and vegetable stalls.
Other things to note
• We have flexible check in and check out times. We try and adjust our times to suit your needs....so just let us know when you would like to arrive and depart and we will do our best to satisfy your requests.
• We offer free storage and our check in / check out times can be flexible depending on other bookings.
• If you are checking into your apartment between the hours of 9pm and 7am, you will need to travel to the apartment with our trusted taxi driver who will have the keys with him and will show you in. He does not increase his tariff for this service.
On arrival he will let you into the apartment and ask you what time you would like our permanent staff to take you on a tour of the area and to show you the apartment in detail.
• We provide a mobile phone upon request and you buy the credit.
• This is a non smoking house, but you may smoke in the balconies and/or terraces.
• If you would like to bring a pet with you please ask if we can accommodate your pet.
|Sleeps up to 2, 1 bedrooms
|Nearest beach
|Kalamaki 8 km
|Will consider
|Long term lets (over 1 month)
|Car not necessary
|Nearest Amenities
|500 m
|Nearest travel links
|Nearest airport: Athens International Airport 35 km, Nearest railway: Syntagma metro station 300 m
|Family friendly
|Great for children of all ages
|No pets allowed, No smoking at this property
|Internet access
|Central heating, Air conditioning, TV, Wi-Fi available
|Kettle, Toaster
|Cooker, Fridge, Freezer, Washing machine
|1 bedroom, 1 bathrooms of which 1 Family bathrooms
|Double beds (1), Dining seats for 4
|Linen provided, Towels provided
|Balcony or terrace
|Not suitable for wheelchair users


text:
Daily Dish
Popcorn PB&J Kids love the idea of mixing "popcorn" with a lunch box staple, and you'll appreciate how easy this snack is to make. Read More
The BabyCenter Blog
Popular Groups
OT/NAD. Intro
by Plutogirl7 in Debate Team (31,388 members)
Target Registry Party #1 - Most exciting registry item!
by BabyCenter Games in February 2016 Birth Club (17,873 members)
Babywearing Party #3 - Where Babywear?
by BabyCenter Games in Babywearing (33,364 members)
Ramzi- post your pics here!
by RMRN in May 2016 Birth Club (12,472 members)
Babies"R"Us Registry Party #2 - Registry benefits
by BabyCenter Games in April 2016 Birth Club (15,887 members)
Adoption Baby Big Kid Birth Club Buddies Family Life Feeding & Nutrition Fertility Issues... Fun & Entertainment Getting Pregnant Great Debates Grief & Loss Health & Fitness Holidays &...


text:
soy wax
Results 1 - 1 of 1
Coastal Candleswww.coastalcandlesshop.com
Go Natural! I specialize in Palm wax candles, soy wax melts, and aroma bears. I also offer tart burners. Each candle is hand poured, highly scented and made to order to ensure a fresh and very fragrant candle.
Category:More Cool Stuff
On Newsstands Now: Oct/Nov 2015
Trending on BUST
Upcoming Events
|Women of Letters (NYC)
Wed Oct 07 @ 6:00PM -
|New Society (NYC)
Wed Oct 07 @ 7:30PM - 10:00PM
|Industry City Happy Hour
Thu Oct 08 @ 5:00PM - 08:00PM
|The Hum
Fri Oct 09 @ 8:30PM - 12:00AM
|The Hum
Mon Oct 12 @ 8:30PM -


text:
October is National Pizza Month!
It's National Pizza Month and what better way to celebrate than with pizza specials! For the entire month of October, we are offering One Large Plain Pizza for $9, or Two Large Plain Pizzas for $16. Plus tax, toppings are additional. Come in to Forno Pizzeria & Grille soon to order your pizzas!Download PDF


text:
IronPlanet joins social media
Lauren Heartsill Dowdle
| November 30, 2010 |
IronPlanet launched its social media program to stay connected and share information with others in the industry.
From our partners
The Importance of Clean Oil: Looking at the bottom line
This is part one of a three-part series on the importance of clean oil. When it comes to equipment lubricants,…
More From: Industry Briefs
- Parsons Brinckerhoff awarded contract for construction engineering and inspection services for I-4 widening in Florida
- Newmont Mining Company names Metso as supplier of the year
- Blastrac opens Greenville, S.C. location, adds Ricky Norkett and Mike Lancaster to 'Team Blue'
- Michael F. Donnino elected to ARTBA's executive committee
- Winners announced for student video contest on transportation issues
- Volvo Trucks names Göran Nyberg...


text:
Quick Links
Browse these popular wedding categories:
Your Selections:
Sort by:
Show Filter
- 8 items
Related Searches
Related Products
Rope Tealight Holders
Dreamily sail away with party guests thanks to these rope tealight holders. Each tealight holder features a frosted glass votive, ...
Vintage Map Tealight Holders
Invite wedding guests to be a part of your wild adventure with these vintage map tealight holders. These charming tealight ...
Antler Tealight Holders
Your perfectly rustic wedding deserves these chic antler tealight holders. Each tealight holder features a frosted glass votive, antler designed ...
Glitter Tealight Holders
Bring out the bling in the form of glitter tealight holders for your bridal shower, anniversary, or reception. Decadently dripping ...


text:
Most recent works and commentary focus on the "challenges" of modern globalization for the various nations of Africa, discounting the centuries-old influence that globalization has had on the continent, and vice versa. This volume attempts to remedy that lack, exploring the influence of…
Hardback – 2016-02-01
Routledge Studies on History and Globalization
The French withdrawal from Vietnam in 1954 was the product of global pressures and triggered significant global consequences. By treating the war as an international issue, this book places Indochina at the center of the Cold War in the mid-1950s. Arguing that the Indochina War cannot be understood…
Hardback – 2012-06-18
Routledge Studies on History and Globalization
This book contextualizes a globalization process that has since ancient times involved the creation, use, and world-wide movement of song, instrumental music, musical drama, music with dance, concert, secular, popular and religious music. Integral to the process have been political, economic,…
Hardback – 2011-10-17
Routledge Studies on History and Globalization
Globalizing Human Rights explores the complexities of the role human rights played in U.S.-Soviet relations during the 1970s and 1980s. It will show how private citizens exploited the larger effects of contemporary globalization and the language of the Final Act to enlist the U.S. government in a…
Hardback – 2011-09-19
Routledge Studies on History and Globalization


text:
A suspect led police on a less-than-dangerous pursuit through the 100 block of North 92nd Street at 2:36 a.m. Saturday.
According to the Wauwatosa police report:
After trying to pull over a car driving 10 mph over the speed limit with expired registration, the officer was led on a pursuit that reached speeds of 30 mph and involved full stops at stop signs.
The suspect calmly drove with an officer flashing his lights and blaring his siren. When the officer called in and said he was in a pursuit due to expired registration, dispatch advised him to cut off the chase.
Officers came to the vehicle owner's house later and towed the suspected fleeing vehicle. The owner's mother was at the house and refused to answer any questions or let police into her home.
The owner came to the police station the next day and asked where they had taken her car. She was given a ticket as well as information on retrieving her car.
This site uses Facebook comments to make it easier for you to contribute. If you see a comment you would like to flag for spam or abuse, click the "x" in the upper right of it. By posting, you agree to our Terms of Use.
E-mail Newsletter
Your link to the biggest stories in the suburbs delivered Thursday mornings.
Enter your e-mail address above and click "Sign Up Now!" to begin receiving your e-mail newsletter Get the Newsletter!
More from News and Features
- wnt_senior meals_1008
- Wauwatosa Meetings: Oct. 8
- In Our Schools: Oct. 8
- Wauwatosa Police Report: Sept. 27 - Oct. 3
- Bedbugs found in Wauwatosa juvenile court center (1)
- Wauwatosa police to try out body cameras on motorcycle patrols
- Wauwatosa Weekly Planner: Oct. 8
- Nordstrom Mayfair announces gala, late-night party fundraising events
- Block Legal Services opens in Wauwatosa
- DA's office will not bring criminal charges against officers who shot and killed man with sword (1)


text: Nice local BB type accommodation in a quieter, higher area of town. Some rooms with ocean view. The breakfast dining area has very nice ocean view. People are easy to talk to and very willing to share and help. And they speak English!!! To this language-deficient traveler, that was a relief! The hotel was a 5-10 minute walk to small... More


text:
ZZ 2yr old Bay Arabian Gelding Started on ground work he won t be able to be trained until...No Price Listed
Lucky is seventeen hands tall and all legs. He is suitable as a trail horse only. Barefoot, goingNo Price Listed
Coming 8yr old unbroken TB gelding. Tyler is very adorable and seeks attention. He desperately needs a job. He's be best suited with an experienced equestrian…No Price Listed
Silver is a very nice using horse We have used him on the ranch worked cattle and ridden a lot in the mountains He is easy to haul and stands nice for…$3,000
KFPS Friesian filly sired by the amazing Fridse 420 Sport and out of my Feike 395 Sport mare for sale Weaning soon and will be ready to an approved home…$10,000
Jazzy is a exquisite twelve year old golden palomino AQHA gelding with good conformation legs and feet Totally sound and healthy He is very smooth responsive…$7,500
Snickers is a super cute welsh buckskin filly She is very out going and adores attention Snickers is the type of pony you don t find to often She has gorgeous…$600
Joey is a exquisite black and white two year old gelding He is very gentle and ready to train your way He should mature to thirteen hands asking 500$500
Simi Valley, CA


text:
Texas Rangers v Tampa Bay Rays
Pinch runner Freddy Guzman of the Tampa Bay Rays steals 2nd base and... News PhotoAmerican League Baseball,Base,Base Stealing,Baseball - Sport,Florida - US State,Freddy Guzman,Full Length,Gulf Coast States,Horizontal,Inning,Major League Baseball,Scoring,Sport,St. Petersburg - Florida,Tampa Bay Rays,Texas Rangers,Tropicana Field,USAPhotographer Al MesserschmidtCollection: Getty Images Sport 2013 Getty ImagesST. PETERSBURG, FL - SEPTEMBER 18: Pinch runner Freddy Guzman #43 of the Tampa Bay Rays steals 2nd base and later scores the tying run in the11th inning against the Texas Rangers September 18, 2013 at Tropicana Field in St. Petersburg, Florida. The Rays won 4 - 3. (Photo by Al Messerschmidt/Getty Images)


text:
UAE: Biggest climber in visa index
Click here for the top 10 rankings in the regionOctober 8, 2015 6:09
Register for our free newsletter
MIDEAST STOCKS-Markets to rise on stronger global environment
Abu Dhabi's ADWEA in talks with banks for $350 mln project bond
Emirates hints at no Dubai Airshow orders, pushes decision on A350/787 to next year
UAE minister says plans to boost oil output capacity on track
MIDEAST STOCKS-Saudi, Egypt markets barely move in early trade
Qatar Q2 GDP growth accelerates to 4.8 pct year/year
UAE says reforms will better protect foreign workers
United Arab Bank secures $125 mln three-year loan
MIDEAST STOCKS-Saudi hit by weak oil, commodities; Egypt down modestly
MIDEAST STOCKS-UAE markets narrowly mixed in thin, early trade
Offerings Provide Businesses with Reliable Disaster Recovery Solutions for Critical Workloads that Enhance Remote Office Backup/Recovery Capabilities
November 16, 2009 12:00 by Aarti Nagraj
Key Facts:
About Double-Take Backup
Double-Take Backup's Features
– Double-Take Backup offers continuous full server protection (OS, applications and data), eliminates backup windows and allows protected data to be sent efficiently over WAN connections to a central location.
– Double-Take Backup combines continuous replication with the ability to recover to any point-in-time (CDP).
– Double-Take Backup makes it easy to recover from disasters of any size by allowing IT administrators to recover anything from individual e-mails to entire workloads (OS, applications and data) either to the original server, new physical hardware or to a VM running on VMware ESX or Microsoft Hyper-V.
– Further differentiating itself from other solutions on the market today, Double-Take Backup includes file archiving capabilities with its Double-Take Cargo™ feature, allowing organisations to reclaim under-utilised storage in branch offices and speed the time it takes to recover after a loss.
About Double-Take Availability
Double-Take Availability's Features
– Double-Take Availability now provides support for VMware ESX v4 and Microsoft Windows Server 2008 Hyper-V R2 and the ability to access the most up-to-date data immediately following a disaster, maintenance or downtime.
– The product's monitoring capabilities now include application-specific intelligence, allowing users to determine if application services are "alive", and attempt to take corrective action and initiate failover based on administrator-established policies. Support is included for popular applications like Exchange, SQL and SharePoint and the feature can be easily extended to support any application running on Microsoft Windows.
– Application Manager Support for Cluster to Standalone protection for Microsoft SQL automates the configuration and management of SQL cluster protection to a non-clustered system at the DR site. This allows administrators using SQL Server in this way to take advantage of Application Manager capabilities like pre-flight checking and target data verification.
Laura DuBois, Analyst at IDC: "Double-Take Software is making it simple for businesses to have comprehensive disaster recovery and business continuity plans in place with its Workload Optimisation suite of products. By combining its existing features and functionality with new updates, the company is providing a compelling suite of products for data migration, flexibility, backup and availability."
Dean Goodermote, President and CEO of Double-Take Software: "Double-Take Software is continuously innovating to bring new approaches to disaster recovery. Double-Take Backup and Double-Take Availability provide businesses with important functionality that will help them protect against disaster and round out a suite of products that play essential roles in disaster recovery and business continuity."
Hard Rock Hotels in Abu Dhabi and Dubai delayed
November 15 2009
Death of the dollar
November 16 2009


text:
Create a free account to check out faster, store multiple shipping addresses, view and track your orders in your account, and much more.
Please enter your password to continue.
This eyelid primer is the Thelma to your eye shadow's Louise. The velvety formula guarantees that colorful shadows stay bright and smudge-free for hours. Suitable for all skin types and tones, the sheer formula blends in seamlessly and doesn't leave any kind of residue on your lids.
I noticed after I first applied to my eyelids the product seemed a little oily. My first thought was that my eyeshadow will end up creasing. I was wrong and end up loving this product. My eyeshadow stays on for the whole day without creases. Also, eyshadow glides on more evenly.5
This eyelid primer is fantastic! At first I thought it would be too oily for my shadows and end up creasing but nothing could be further from the truth! It dries dry and smooth - no oily feeling. I have used MAC's eye primer for about a decade and it would almost always crease when it dried. This works just as well, if not better, and I need barely use a pin-drop-sized amount for each eye.5
I had my doubts- I have used drugstore primers before without much success. However, its now 2pm, and a solid 8 hours after I put makeup on this morning, I still look AWESOME. Most of my eyeshadow and liner has usually floated down into undereye circles by this time in the day- it's all still exactly where I put it this morning, and still looks fresh.
I have relatively sparse brows, and what I really appreciate about this is that it helps my brow pencil and powder stay put too! I will definitely purchase!!


text:
Thanks for helping us associate the correct Pornstars to this video!
Help Us: Do these Pornstars appear in this video?
Pornstar Name:
Thanks for helping us sort this video!
All Comments (28)
i need a cock like that to suck
im straight............but i love this cock !!!
Well she sure knows how to suck dick, I'm fucking happy just watching her work!
AW that is you !!!!
So weird, this girl is my ex's exact double.. or.. wait a minute..
Wow that is a big cock,I love the big veins buldging from it.
Anybody knows who they are? this girl is fantastic, i wish to know her name, any help? thanks
love the way it is being done i love sucking on big cocks too
I want a big cock to suck on long,,wet,,hot sucking then feed me you're hot cum,,mmmm
I wanna see one where they suck the dick and balls at the same time
how shes name Magda maybe??
i loveeeeeeeee this cock
i'm not gay, but i would try to suck a cock of that size one day, it turns me on so much
this girl sucks with a lot of love
Now that`s what I call a cock sucker.LOL
mine is about the same size but my gf is a better sucker. She gets all wet she when she feel it big anf fat in the hands and mouth. I love to mouthfuck her!! good video do later will show it to her to compare. ..and her slurping my dick while she reading about herself doing it :D
agree wouldnt mind if i was her or him...either way would be amazing...slurping that giant cock and taking the warm load or getting my rod massaged until i blow up
Seconded. I wouldn't care if I was her or him, that's a good sucking he's getting and that's a *great* cock she's sucking...
i am so happy
this is a big cock!! & this is a good cock sucking!!! http://www.youporn.com/watch/17024/deepthroat-a-big-cock/?from=related3&al=2&from_id=2907&pos=1
She is amazing... I need a blowjob like that
Good penis, good sucker. Definately hot. Not enormous, huge or even big cock, it's decent. I don't love the slow tempo. Cumshot was kinda disappointing. Over all good clip. Lol @ comments.
Lol at that loser's comment^^^^
bonne grosse queue ! heureux homme !
contact me u can have mine
I wish I was in her position. Oh my god I´m bi-sexual!


text:
Lexmark Genesis S815
Excellent print quality, useful online resources, impressive touchscreen, super-fast scans
Very slow print speeds, expensive
It's not often you see something a bit different in the printer/AIO market but Lexmark has produced a device that should turn a few heads in the Genesis S815. Most immediately striking are the aesthetics, since it utilises an innovative vertical design finished with a sleek piano-black front panel that also houses a 4.5-inch colour touchscreen.
Lexmark isn't going all "super-model" on us and focusing the appeal of the S815 solely around looks however, as it boasts a "world's first" technology called Flash Scan, which can allegedly scan images in 3 seconds, as well as instantly previewing the page on the main display for adjustment. You'll also find more typical conveniences such as wireless printing, a PictBridge USB port, multicard reader, duplex mode and more current advancements in the form of access to online content via the touchscreen display.
All of this sounds very good, but ticking the boxes in terms of key performance is obviously a must. Sadly it doesn't cover all its bases here, as we found print speeds to be far, far slower than the 33ppm quoted draft setting. In our tests we found 8ppm to be more accurate; a significant drop, and at regular setting the quoted 18ppm was more realistically narrowed down to 5ppm. Similarly printing full colour photos takes longer than we'd expect at over 3 minutes for a borderless A4 print and over a minute for a 7 x 5-inch.
Thankfully this slow speed is made up for somewhat by some excellent quality results. Photos in particular looked stunning from the S815, with accurate colours, distinctive contrast and sharp lines. Text was also crisp and clean with no bleed or banding even at draft setting, so if professional quality results are your aim, this device can certainly deliver.
It was also pleasing to see that the flash scan technology wasn't as overstated as the print speeds. When a page is loaded into the device it is instantly "snapped" and displayed on the colour screen, with some basic photo editing options available to customise the image before scanning. It did indeed take just 3 seconds for an image to be rendered and a save dialog pop-up on our computer. Scan quality isn't top-notch however, and we noticed a distinct grain over images that puts its ultimate performance in this area below similar mid-range devices, despite the outstanding speeds.
The final benefit on offer from the S815 is access to online content via the device's SmartSolutions widgets. These are effectively a collection of resources that can be included in a customisable list of shortcuts and accessed directly via the printer. These are sorted into a range of categories and include benefits such as printable calendars, sheet music, graph paper and other forms and templates, informational resources such as weather, RSS feeds and clocks and various others to help improve productivity. This all worked very well in our tests, and confirmed our conclusion that unlike some other rival devices the touchscreen is nicely responsive, colourful and easy to use for both access to SmartSolutions and general operation of the device.
When you consider the print speeds alongside a relatively meagre 100-sheet A4 tray the S815 seems clearly oriented towards the home market rather than business use. Nothing wrong with that, but even despite the excellent output quality and range of additional benefits this is still a lot to pay for what is effectively a mid-range all-in-one. If print speeds had been up to scratch it would have been easier to justify the outlay, but as it stands this is a hefty investment considering the alternatives available.
The Genesis S815 is a great looking device capable of some excellent quality output, a super-fast scan feature that actually works and additional benefits in the online content available. £400-ish is still a lot to pay though, and since it loses points in some key areas we think it'll struggle to really take off.
Harry Potter books get new animated life for Apple iBooks Store release
EE Pocket-lint Gadget Awards long list goes live: Vote for your favourite gadgets today
HP takes on Apple Music and Spotify with HP Lounge, free unlimited music streaming
Master the CompTIA advanced security practitioner certification exam
- Harry Potter books get new animated life for Apple iBooks Store release
- EE Pocket-lint Gadget Awards long list goes live: Vote for your favourite gadgets today
- Master the CompTIA advanced security practitioner certification exam
- Master HTML5 and CSS3 web design for 81 per cent off
- You choose the price for 12 Microsoft Office courses
- Secure your digital life with private internet access VPN: Save 24 per cent
- Mystery box for geeks: Get a 3-month subscription to Loot Crate
- Google Nexus vs Google Pixel: What's the difference?
- Watch the Google Nexus live stream
- Master big data with the Ultimate Data & Analytics bundle (97 per cent off)
- Best smartphones 2015: The best phones available to buy today
- Microsoft Lumia 950, Lumia 950 XL vs Apple iPhone 6S, iPhone 6S Plus vs Google Nexus 5X, Nexus 6P: What's the difference between the flagships?
- HP takes on Apple Music and Spotify with HP Lounge, free unlimited music streaming
- Castrol Nexcel gives you a 90 second engine oil change, debuts on Aston Martin Vulcan
- Apple iPad Pro vs Microsoft Surface Pro 4
- Humax unveils the first Freeview Play set-top box
- Dell XPS 15 adds infinity display for 2016 model, XPS 13 sees power bump
- What is Freeview Play, when is it coming and how can I get it?
- DJI Osmo is basically a pricey selfie stick that can shoot steady vids in 4K
- Chromecast Audio review: Simply connect your disconnected speakers


text:
Tel: 01506 436666 Email: firstname.lastname@example.com
05 October 2015
Join West Lothian Credit Union
- Free Savings & Loans Insurance
- Friendly and flexible
- Easy way to save
- Access to low cost loans
- No bank charges or hidden fees
- Affordable and ethical
- Working in your local community
What kind of a loan do you need?
- Every Week Total Total Interest Payable
- Every Month Total Total Interest Payable
Whilst every care has been made in the production of this document, the Credit Union, or its staff, cannot be held responsible for any omissions, errors or other mistakes made herein. This document is for illustrative purposes only, so as to give you, the borrower, an overview of the potential cost of borrowing.
Save with Us
Build up your savings with regular deposits through your wages, over the counter or by standing order.
Online Banking
You can view your account balances, transfer money to your external bank account or pay your bills all from the comfort of your home.
Loan rates from 3% APR (Representative rate 12.68% APR).
• Tell a friend - Competition Draw! And you could win £20. Introduce a new member to West Lothian Credit Union and your name will be added to the Competition Draw for Free! Remember and provide your details so that you can be entered for the draw • ARE YOU THINKING OF ANY HOME IMPROVEMENTS THIS YEAR? Why not apply for a loan from West Lothian Credit Union rates from 3% APR (Rep Rate 12.68% APR) • We Are Looking For Volunteer Directors Do you have the Skills?


text:
Hakone Hisui, Hakone
- Free Internet
Main amenities
- Restaurant and bar/lounge
- Hot springs
- Breakfast available
- Sauna
- Garden
What's around
- In Sengokuhara
- Lalique Museum Hakone (0.3 mi / 0.5 km)
- Museum of the Little Prince (0.3 mi / 0.5 km)
Enter dates to check prices
Hotel Info: 020 3027 9803from £443
- Deluxe Room, Non Smoking
Also popular with travellers from the UK
At a glance
Key facts
- Check-in time 3 PM-8 PM
- Check-out time is 11 AM
Required at check in
- Credit card or cash deposit required
- Government-issued photo ID required
Travelling with others
- Pets not allowed
- Free wired Internet access in public areas
- Free wired Internet access in rooms
- Accessible parking
Nearby Four Star Hakone hotels
In the hotel
- Full breakfast daily (surcharge)
- Restaurant
- Bar/lounge
- Room service (during limited hours)
- Sauna
- Elevator/lift
- Garden
- Disabled parking
- In-room accessibility
In the room
- Climate control
- Minibar
- Iron/ironing board (on request)
- Private spa tub
- In-room massage available
- Patio
- Flat-screen TV
- Desk
- Free wired high-speed Internet
- Phone
- Refrigerator
- Free bottled water
- In-room safe
Top Hakone hotels
Recently reviewed hotels in Hakone
- Arai RyokanFrom
- Highland Resort Hotel & SpaFrom
"The room was nice with a good view of Mount Fuji. It is worth asking for a room with this view. But the hotel was very expensive for what it was. We didn´t realise until we got…"Aug 24, 2015
- Manatei HakoneFrom
"This is a fantastic spa hotel tucked away in the mountains. You arrive at a local train station, and the hotel will come pick you up. They will make you comfortable, and then…"Aug 20, 2015
- Shoheiso RyokanFrom
"Fantastic food and service. Friendly, helpful staff."Aug 5, 2015
Nearby hotels
Special features
DiningICHIYUU - This restaurant serves breakfast and dinner. Reservations are required.
Hakone Hisui, Hakone's small print
The Japanese Ministry of Health, Labour, and Welfare requires all international visitors to submit their passport number and nationality when registering at any lodging facility (inns, hotels, motels, etc. ). Additionally, lodging proprietors are required to photocopy passports for all registering guests and keep the photocopy on file.
Reservations are required for massage services and spa treatments and can be made by contacting the property before arrival at the number on the booking confirmation.
Mandatory fees
You'll be asked to pay the following charges at check in or check out:
- A tax is imposed by the city: JPY 150 per person, per night. This tax does not apply to children under 13 years of age.
Optional extras
Cribs/Infant beds are available for JPY 2100 per night
We have included all charges provided to us by this hotel. However, charges can vary, for example, based on length of stay or the room you book.