Technical Guidelines for the GCS Petaflop Supercomputer JUQUEEN in Jülich

Important: The following requirements must be fulfilled for a project on JUQUEEN:
  • At least 5 mio core-h should be requested. If less are requested, special reasons have to be given why this project can only run on JUQUEEN.

  • Give a clearly arranged work schedule using a table and/or Gantt chart.

  • Make sure that the amount of core hours you request can be used up with the job sizes you aim to use and the number of jobs which can be run simultaneously.

  • Please be aware that one node consists of 16 cores with four-fold SMT and is equipped with 16 GB, i.e. each physical core has at most 1024 MB of main memory available. In order to use the architecture efficiently, pure MPI codes must use at least 32 tasks per node, hybrid codes must use the corresponding number of threads per task (from 32 to 64 threads per node). Therefore, pure MPI applications must use less than 512 MB per MPI task, hybrid codes not more than 16 GB per node in order to be suited for the architecture.

  • The minimum number of (physical) cores per job is 2048. However, it is expected that GCS projects applying for this system can use more than 2048 physical cores per job on average (at least 4096 physical cores). Job sizes must use a multiple of 2048 physical cores in order to make efficient use of the architecture.

  • The project should be able to start immediately and is expected to use the resources continuously (a maximum of 1 month at the beginning of the project for fine tuning of the parameters or the application must not be exceeded). Please state clearly in your proposal if you need such fine tuning and which amount of time will be necessary.

  • When requesting more than 20 TB of scratch disk space and/or storage of more than 4 mio files, please justify this amount in your proposal. If you request more than 100TB of disk space, please contact sc@fz-juelich.de before submitting your proposal in order to check whether this can be realized.

  • All data must be removed from the execution system after the end of the project. Please state clearly in your proposal the amount of data which needs to be transferred after the end of your project to your local system.

  • Please be aware that the transfer of 100 TB or more from the production system at the end of the project is not feasible due to limitations in bandwidth and time. Larger amounts of data have to be transferred continuously during project’s lifetime.

  • Not more than 10 TB/week should be moved to it.

  • I/O intensive applications need to read and/or write frequently from/to disk during a job. In case your application is I/O intensive, please describe the I/O strategy of the code. In general serial I/O for large amount of data is not suitable for the system.

  • Please provide explicit scaling data of the codes you plan to work with in your project up to 8192 physical cores using input parameters comparable to the ones you will use in your project (a link to external websites, just referencing other sources or “general knowledge” is not sufficient). Generic scaling plots provided by vendors or developers do not necessarily reflect the actual code behavior for the simulations planned.