|
|
@ -1061,10 +1061,9 @@ chosen and implemented.
|
|
|
|
|
|
|
|
|
|
|
|
#### Caching mechanism
|
|
|
|
#### Caching mechanism
|
|
|
|
|
|
|
|
|
|
|
|
As described in fileserver section stored supplementary files have special
|
|
|
|
Worker can use caching mechanism based on files from fileserver under one
|
|
|
|
filenames which reflects hashes of their content. As such there are no
|
|
|
|
condition, provided files has to have unique name. If uniqueness is fulfilled
|
|
|
|
duplicates stored in fileserver. Worker can use feature too and caches these
|
|
|
|
then precious bandwidth can be saved using cache. This means there has to be
|
|
|
|
files for some while and saves precious bandwidth. This means there has to be
|
|
|
|
|
|
|
|
system which can download file, store it in cache and after some time of
|
|
|
|
system which can download file, store it in cache and after some time of
|
|
|
|
inactivity delete it. Because there can be multiple worker instances on some
|
|
|
|
inactivity delete it. Because there can be multiple worker instances on some
|
|
|
|
particular server it is not efficient to have this system in every worker on its
|
|
|
|
particular server it is not efficient to have this system in every worker on its
|
|
|
@ -1076,7 +1075,7 @@ exactly needed. But mainly it would be single-failure component if it would stop
|
|
|
|
working it is quite problem. So there was chosen another solution which assumes
|
|
|
|
working it is quite problem. So there was chosen another solution which assumes
|
|
|
|
worker has access to specified cache folder, to this folder worker can download
|
|
|
|
worker has access to specified cache folder, to this folder worker can download
|
|
|
|
supplementary files and copy them from here. This means every worker has the
|
|
|
|
supplementary files and copy them from here. This means every worker has the
|
|
|
|
\possibility to maintain downloads to cache, but what is worker not able to
|
|
|
|
possibility to maintain downloads to cache, but what is worker not able to
|
|
|
|
properly do is deletion of unused files after some time. For that single-purpose
|
|
|
|
properly do is deletion of unused files after some time. For that single-purpose
|
|
|
|
component is introduced which is called 'cleaner'. It is simple script executed
|
|
|
|
component is introduced which is called 'cleaner'. It is simple script executed
|
|
|
|
within cron which is able to delete files which were unused for some time.
|
|
|
|
within cron which is able to delete files which were unused for some time.
|
|
|
|