High Performance Computing
HPC Computing Facilities
General Purpose Free Facilities
Gemini.science.uu.nl Unix server. Free to use but the machine is shared by many users. This computing facility provides a good solution for the development and testing of small program that doesn't require huge dedicated computing power.
The general characteristics of the machine are: 2 nodes, 32 cores per node, OS : Scientific Linux 7.7 {equal to Redhat EL 7.7}, serving short and long queue, for batch jobs, a lot of standard software, easy to use through environment modules, limited interactive jobs possible, local scratch storage for fast i/o jobs.
Characteristics (+) Easy to use and access. (+) Efficient and effective for small interactive jobs or batch processing (+) Fast local disk space (+) Free (-) No configuration possible (no root user access) (-) No exclusive access (-) No cluster jobs, as this is a single machine (-) No GPUs, thus no visual computing
Contact: science.ictbeheer@uu.nl
Dedicated Virtual Machines
The Faculty has a VMWare installation and can provide virtual machines (VMs) on demand on a pay per use basis. The memory and the space of these machines depends on the specific requirements. Some indicative costs are the following:
• 8 cpu cores with 16 GB: 40 euro per month • 8 cpu cores with 64 GB : 64 euro per month • 8 cpu cores with 128 GB: 96 euro per month • 100 GB local disk space, 2 euro per month (not for archive) • 1 TB, 2 euros per month (for long term storage/archive). For more storage options, see https://www.uu.nl/en/research/research-data-management/tools-services/tools-for-storing-and-managing-data/storage-solutions
Users can have full administrative rights and exclusive access on these machines. They are for CPU computation and their characteristics can be easily adapted according to the needs of the computation.
(+) Easy to use and full access. (+) Good for either interactive or for batch jobs (+) Machines can be configured as from scratch as needed (installation of software etc.) (+) Exclusive access (+) Can be used for cluster-based/parallel jobs. (-) Although they can be used for parallel jobs, they are not the best option, since, being virtual, there is little control over the usage of the different cores. Hence, precise timing measurements cannot be done at core level. (-) Users need to configure these machines from scratch (-) Disk is virtual which means that is not ideal to measure I/O operations (-) Cost (-) no GPU/visual computing support
This option provides a good solution if one needs exclusive access and special software to be installed. Also, having a VM can be turned on/off on demand, thereby offering flexible cost management. This flexibility also means that one can easily switch from cluster to single machine computation easily.
Group Clusters
Different research groups have bought over time (e.g. in the context of a project) servers that work individually or in the form of clusters. Since such machines were based on different needs, they are widely differing in resources (#nodes, #memory, #cores, #architecture, #interconnects).
(+) Fully customizable machines (by their owners) (+) May offer GPU/visual computing (+) Physically accessible (- ) Access not guaranteed via a formal scheme (up to the latitude of the owner) (- ) Not suited for large-scale computations
3.4 UBC Cluster The UBC cluster is an HPC facility located in UMCU. It is part of the Utrecht Bioinformatics Center (UBC). It can be used by any researcher at the Utrecht Science Park. It has more than 2000 CPU cores, 30 GPUs, and 20TB of RAM. 1.7 PB high performance storage and 1.1 PB archive storage running CentOS Linux. It is controlled by the SLURM batch-wise querying system with a few head nodes and data transfer nodes. It is both a CPU and GPU Computing Cluster. Some information can be found here: https://ubc.uu.nl/infrastructure/high-performance-computing. The UBC cluster is free for trial usage. For actual usage, a research group buys shares. Priority is given to researchers with more shares. Each group has its own “space” that facilitates sharing among members of the group. Packages can be installed (based on Guix for library management).
Indicative usage costs CPU: 1200 EUR for 50K CPU hours GPU: 1150 EUR for 5K GPU hours (+5 cores 64 GB ram). One can use any available GPU. Storage: 180 EUR per TB per year Archive: 45 EURper TB per year
(+) HPC, so it is for very fast computation (-) Relatively expensive storage (compared to ITS machines)
Given all above, this solution is preferable for large-scale experiments of ready-made programs (used on a pay-per-use mode) and not for actual development or testing of a program.
Contact person
Ies Nijman (inijman@umcutrecht.nl) 3.5 SURFSara SURFsara offers an integrated ICT research infrastructure and provides services in the areas of computing, data storage, visualization, networking, cloud and e-Science. It is the national computer center for scientific research. It has two supercomputers: Cartesius and Lisa. They are both production systems. Not development. The former is for extremely computation intensive jobs. They support both CPU and GPU computations. They use a model of credits. One can submit a project proposal and get credits to use the system (for free as long as the proposal is accepted). If it is for less that 50K core hours, the proposal can be submitted directly to Surf Sara. Instead, if it is more, it is submitted to the NWO. Each machine has a 200GB scratch space that is NFS. There is also an archive service which is backed up and can be at the level of TB. There is also the option of a project space, but it is also scratch and not backed up.
In some sense, SURSara operate like the external providers, but they are more economical and more accessible. More information on this can be found on the SURF Sara web site https://userinfo.surfsara.nl. The researchIT department can help into selecting and setting up a Surf Sara set of virtual machines. It can also support it financially for the first use to try (see: https://www.uu.nl/en/news/limited-time-free-computing-power-for-researchers. Then it becomes a pay-per-use model. To create an account, different options exist: https://userinfo.surfsara.nl/systems/shared/rccs.
(-) free (as long as the underlying project is approved) (-) production machine only (+) professional support (remote)
Contact persons
The RDM team (www.uu.nl/rdm) or directly Surf Sara (helpdesk@surfsara.nl)
3.6 External Private Providers
There are traditional providers in the market that offer computing services, e.g., Amazon, MS Azure, Baidu, and Google Colab. The Research Data Management department can provide guidance on how to obtain and set up cloud-computing services like AWS, Azure, and Google Cloud. There are no free Sara credits to try the machines at the beginning. However, some providers offer partially-free services, e.g. Google Colab. These machines offer mainly a Jupiter notebook service, while some others are basic system level offerings of machines (VMs) to be configured as desired. This of course requires admin knowledge.
The problem is that this cannot be a temporary solution. One cannot repeat the procedure every time. These facilities may be a good alternative if a collective solution is provided for long term, e.g., the way the university has chosen to use Microsoft or OneDrive services.
Some indicative prices for the Azure service, for instance, can be found here: https://azure.microsoft.com/en-us/pricing/details/virtual-machines/linux/
• 4 cpu cores with 16 GB: 86 euro per month • 16 cpu cores with 64 GB : 345 euro per month • 32 cpu cores with 128 GB: 690 euro per month • 100 GB local disk space, 16 euro per month (not for archive)
(-) not free (and maybe more expensive that local NL or local UU solutions)
(-) cannot be used as a partial solution
(-) no direct support from the UU technicians/services
(+) professional-level support (paid)
Contact person:
The RDM team (www.uu.nl/rdm)
3.7 Own machines Researchers can always buy own (personal) machines. These can be hosted at the ITS data center and they remain in exclusive property of the purchasing researcher. A positive aspect is that most costs are paid upfront and not ‘as you go’. Administration can be done by the owner or can be outsourced to the ITS at a basic cost. Such a machine could be completely autonomous or attached to the Gemini grid mentioned above.
(+) Full administrative control over the machine (+) Full control over the CPU, GPU, and disk access (+) Can support GPU/visual computing (-) Potentially large upfront cost (-) For long-term usage (5..10 years or even more), hardware upgrades are needed, which incur additional costs, and may cause compatibility problems (-) Administrative expertise required
Contact persons:
Marjon Engelbarts (M.Engelbarts@uu.nl) Carel van der Werf (C.G.vanderWerf@uu.nl)
3.8 Support There are different services that can offer support throughout the university.
1. The direct contact people mentioned at the end of the above sections can provide first-order information or forward to other people for extra information. 2. The Research Data Management Service (https://www.uu.nl/en/research/research-data-management short url: www.uu.nl/rdm) of the UU operates a team for supporting researchers in their various needs. The service runs by the Research Engineering team (https://www.uu.nl/en/research/research-data-management/tools-services/research-engineers) 3. A service that is currently in preparation is the Digital Research Environment. More Info can be found here: https://www.radboudumc.nl/en/research/radboud-technology-centers/data-stewardship/digital-research-environment
Support seems, however, to have some issues. Several of these are listed below:
• There is no central facility service that can answer high-level user questions (see Q1 and Q2 in Sec. 1) and can direct further requests to the right service provider with some response time guarantees. Specific people in different groups have some experience but it is often by word-of-mouth that are found. Furthermore, since they may work in different departments/groups/organizations, they may not have the time or obligation to deal with the specific requests they receive. • Related to the above, there is nowhere a comprehensive list of all offered facilities and the respective contact points. The descriptions that are currently found on the internal pages are quite generic and for the general public. Few technical details are provided. Hence, answering Q1 and Q2 by an individual researcher is hard, even after consulting the separate descriptions provided on their respective webpages.
3.9 Other Auxiliary Tools
There are also other tools that are not related to computing but are useful to the data studies. Just as a reference, Shiny Apps, Qualtrics, etc. or the https://intranet.uu.nl/en/software-work-teaching-rooms-and-home and the https://www.uu.nl/en/news/new-survey-tool-for-uu-staff-qualtrics