Dear Grex Users,

It is time to renew the allocation of Grex resources as well as inform you of changes to Grex hardware.  Please see below and attached.

Thank you,

j

John (Jay) Doering, Ph.D., P.Eng., FCSCE, FEC, FCAE Associate Vice-President (Partnerships), Professor of Civil Engineering, & Dean Emeritus Office of the Vice-President (Research & International) 202 Administration Building University of Manitoba Winnipeg, Manitoba, Canada  R3T 2N2 Tel: (204) 474-9568  |  Fax: (204) 261-1318

This electronic mail (e-mail), including any attachments, is intended only for the recipient(s) to whom it is addressed and may contain information that is privileged, confidential and/or exempt from disclosure. No waiver of privilege, confidentiality or any other protection is intended by virtue of its communication by the internet. Any unauthorised use, dissemination or copying is strictly prohibited. If you have received this e-mail in error, or are not named as a recipient, please immediately notify the sender and destroy all copies of it.

_______________________

Overview

Grex was put into production on May 2^nd, 2011.  Its current compute capacity consists of three parts: the legacy compute nodes, the new nodes purchased in 2020 and 2021, and user-contributed nodes. These parts are as follows:

· Legacy compute from 2011

o 316 original compute nodes, each 12 cores 2.67GHz processors (Intel Westmere architecture) with 48 or 94 GB of memory, non-blocking 40GB/s QDR Infiniband Interconnect

· 418TB of Parallel, scalable Lustre storage

· New compute nodes added in 2020 and 2021

o two GP GPU nodes (Intel 5218 Cascade Lake, 4x NVIDIA V100)

o 12 of 40 core Intel CascadeLake 6248 2.5GHz compute nodes, 384GB RAM

o 42 of 52 core Intel CascadeLake 6230R 2.1GHz compute nodes, 96GB RAM

o the new compute nodes have 4:1 blocking EDR (100GB/s) interconnect

· Contributed GPU nodes

o 3 of  4x V100 nodes

o 1 of 16x V100 HGX-2 node

In this round of resource allocations, we will allocate newly added compute nodes only (total of 2664 CPU cores, 54 CascadeLake nodes).

· General GP GPUs nodes available on a first come, first served basis. Please indicate if you might need GPUs in your proposal.

· We plan to decommission the legacy 316 compute nodes at the end of October 2021. Until that date, they are available on a first come, first served basis. Please indicate if you need these nodes in your proposal.

· Contributed GPU nodes are not allocatable, they are available for opportunistic, preemptible use when not used by the contributors.

Proposals for the use of Grex resources will be reviewed by the Advanced Research Computing (ARC) Committee.  They will also be reviewed by Grex technical staff to ensure that Grex resources will be used appropriately and efficiently.

Categories of Resource Requests

The request for ARC resources must come from a PI.  There are two (2) categories of resource requests:

i.) Rapid Access Service (RAS)

Is limited to less than: 30 core-years, 4 GB memory, and 4TB of storage per research group.  For information and access please email arc@umanitoba.ca

ii.) Resource Allocation Competition (RAC)

This includes resources in excess of those allocated for RAS and requires completion of a proposal in the format outlined below.

Proposal Format

Please use the attached section headings for your proposal, leaving 2.5 cm (1.0 in) in all the margins and using Times New Roman font (12 pt) with single spacing between lines. You may remove all explanations (text in italics) if desired. Please ensure that all pages of your proposal are numbered. The proposal document must be submitted in pdf format only.

This template attached is intended as a guide. Please remove any section or sub-section that does not apply to your application.  For example, if you are not requesting a storage allocation, there is no need to include section 3.2.  Maximum lengths for each section are included in the template, but your proposal may use (much) less space depending on the size and complexity of the request.

Proposals should be sent by email to ARC@umanitoba.ca by 4:30 pm May 5, 2021.