Computation modules
Files play a big role in Hoopoe, as they contain data to be processed by tasks
users submit.
For this purpose, Hoopoe supports several storage mechanism providing a high
level of flexbility.
CUDA
HoopoeFS is a native service in Hoopoe, providing a simple storage service. The
API supports basic file-system objects (files, directories etc.) and was created
with simplicity in mind so users can take care of the computing.
For example, working with files and directories support the very same
operations:
- Create
- Delete
- Rename
- Get information
Where files also support I/O operations:
HoopoeFS has no limitations for the size of files a user can upload. All
functions in the API are meant for 64 bit support to work with very large files.
One big advantage of using the HoopoeFS is that the storage is placed in the
cluster, so it doesn't require further data transfers after the users uploaded
the files.
FFT
Amazon S3 is the storage service provided by Amazon and is supported in Hoopoe
for convenience of users.
When submitting a task, the user provides further information about the origin
of the buffer, where it is possible to enter the exact path of the file under
S3.
Hoopoe can also write data files to the Amazon S3 service for every task,
following the same manner described in the previous paragraph.