From a3fc1c69a7de8dd3127ae35684be5f8fd11e720c Mon Sep 17 00:00:00 2001 From: Martin Polanka Date: Mon, 9 Jan 2017 16:58:04 +0100 Subject: [PATCH] Leave out fileserver hashing from worker desc --- Rewritten-docs.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/Rewritten-docs.md b/Rewritten-docs.md index dc56e24..fa0fa21 100644 --- a/Rewritten-docs.md +++ b/Rewritten-docs.md @@ -1061,10 +1061,9 @@ chosen and implemented. #### Caching mechanism -As described in fileserver section stored supplementary files have special -filenames which reflects hashes of their content. As such there are no -duplicates stored in fileserver. Worker can use feature too and caches these -files for some while and saves precious bandwidth. This means there has to be +Worker can use caching mechanism based on files from fileserver under one +condition, provided files has to have unique name. If uniqueness is fulfilled +then precious bandwidth can be saved using cache. This means there has to be system which can download file, store it in cache and after some time of inactivity delete it. Because there can be multiple worker instances on some particular server it is not efficient to have this system in every worker on its @@ -1076,7 +1075,7 @@ exactly needed. But mainly it would be single-failure component if it would stop working it is quite problem. So there was chosen another solution which assumes worker has access to specified cache folder, to this folder worker can download supplementary files and copy them from here. This means every worker has the -\possibility to maintain downloads to cache, but what is worker not able to +possibility to maintain downloads to cache, but what is worker not able to properly do is deletion of unused files after some time. For that single-purpose component is introduced which is called 'cleaner'. It is simple script executed within cron which is able to delete files which were unused for some time.