Concept Design Update Meeting¶

Created maanantai 19 elokuu 2024

Obs: Text in square brackets [ like this ] were not discussed in the meeting but were added my be afterwards.

Space¶

• Location and space use as defined previously
• Fire safety, why is P1 needed in data center space? / Verify with fire safety
• What fire safety in Telecom room?

Heat management and reuse¶

• Interface to the building and heat reuse, we need connection to BMS of Technobohtnia (See AP4)
• Heat rejection in summertime with dry cooler
• Cooling, Liquid and air (when liquid cooling is used heat rejecttion mostly with water pipes)

• Rittal Liquid Cooling Package, LCP plus (not direct to chip)

Video about cooling solutions https://www.youtube.com/watch?v=4Np1HnWiHb4
Gigabyte Direct To Chip cooled server https://www.gigabyte.com/Enterprise/GPU-Server/G363-ZR0-LAX1

Batteries and power supply¶

• Telecom room (teletila) for batteries
• Batteries: FePO is already used a lot, NaI is the next big thing
• 400 VDC power supply, cables needs to be installed
• AC/DC converters are available for example from [[https://zekalabs.com|Zekalabs]]
• VAMK has battery cell test equipment already (Charge-discharge, make the characteristics curve, etc)
• VAMK has Intrest to combine forces with other research units to make a more complete battery research lab.
• We can virtually integrate existing solar panels in the system, no need to install more panels for Wstar
• openEMS: open source energy management system, can be used to controll the use of Batteries. It does no necessarily cover all other automation needs, but we need also other automation system.

Action points, hopefully done already during this week¶

• AP1: Organize meeting with Fire Safety, / Petri**
• AP2: Market research, including Danfoss, Natron, APC, Vertiv, ABB, NorthVolt / Petri, Johan
• AP3: Deatailed power feed drawing / Johan

For procuring the UPS/Power converters we write our requirements to the tender, and if we do not have any offers, we will purchase it separately.

Typical AC power feed

Example of DC power feed

Building automation and IoT¶

• Integration with Technobothnia is needed
• Configurability and open interfaces are mandatory and even open source solution is favourable, because the automation system optimisation is an important part of energy savings and needs to be carried out by ourself
• The HVAC design needs to be updated. For example there is no isolation valves in the current design, [//and it does not allow flexible testing of different scenarios//]
• Using OpenEMS for energy management would be a good solution, works in facility and in rack level
• Some off-the self solution is also possible, if it has open interfaces and easy extendability

• AP4: Organize a meeting with building owner and facility management. How to connect to the building, and who pays the costs of the changes needed in the building / Petri

IT HW¶

SOFOR's proposal: 16 servers with 16 GPUs, 14 kW power dissipation, 400 k€ + ALV
Some air cooled, some liquid cooled and perhaps some experimental 2-phase?
Network and storage?

The amount of liquid cooling GPUs may be limited. We can make our own designs, but it may affect to guarantee and there is also a danger to break them in case of leaks. That needs to be taken into account.

[ I made some googling after the meeting. At least NVidia A100, H100 and H200 GPUs are available as liquid cooled. And for example Supermicro and Gigabyte have several servers with liquid cooling, such as G363-ZR0-LAX1

[ By the Way H100 Grace-Hopper is a very interesting chip. It includes Nvidia's ARM CPU (Grace) and their GPU (Hopper) integrated, Some computer have them packaged in one liquid cooled server, like H263-V11-LAW1 , which is probably too expensive]

Second hand HW would be also great thing to save the costs and be more suitable for testing. And reduce CO2 emissions.

AP: Finish the specification of servers: What servers, liquid / air cooled, CPU architectures (AMD, Intel, ARM, FPGA), second hand servers / Andreas, Petri, Hussain

Procurement and project¶

• Procure construction and IT separately
• Is HVAC included in construction project?

1. Procurement for construction and HVAC < 150 € + VAT
2. Procurement for IT components The rest