[2020] Extended Downtime Notification - 2020/06/24-25

Hello, friends,

We will have a couple of extended downtimes this week to perform a range of database testing. The downtime will be extended on:

• Wednesday, 24 June
• Thursday, 25 June

Each downtime may take up to 30 minutes and we expect Tranquility to be back accepting connections by 11:30 UTC at the latest. However, we will let players in earlier if work is finished earlier. Updated will be posted in this thread and on EVE Status Twitter as they become available.

Thank you for your understanding and sorry for any inconvenience this may cause.

UPDATES:

UPDATES:

We’re excited to test this in production tomorrow!! Anyone care to guess specs??

You can hold to the faith… it has significantly more ram and cpu cores than that

It is a dell chassis, R7xxxx

(post withdrawn by author, will be automatically deleted in 24 hours unless flagged)

some of those numbers are correct!

no, this is testing new db hardware for tq

close enough!! This is a R7525 with a single EPYC 7742

AMD!! We’ll see how 128 cores compares to 32 tomorrow!

The core of the simulation is still on Windows (DB and Sol Nodes). This particular server is Win 2019 - and I hear you with lots left on the table, but not just with linux. SQL Server in general really benefits from tuning to the workload being thrown at it.

we’ll do a dev blog for sure

in the near future we’ll also test more traditional setups - dual socket, 8 core CPU’s from intel/amd - this will match our current 32 cores, with as you say, max cpu performance.

We’ll do a devblog later and have more info on what we’re measuring with more cores/less numa nodes, etc…

And licensing is a giant thing

No, I don’t think so The DB typically isn’t a bottleneck in big fights. But this will likely help with slowness for the first while post-startup and also on busy weekends the current DB struggles.

Market is one of the big things that should be snappier with this type of change

this is 16x128gb modules for 2 tb it would have been 4 tb of memory but the vendor said all 256gb chips were sold out worldwide.

but correct on the dual sockets / unused memory slots

it would of been REALLY interesting to have the other socket populated the same - 128/256 ht cores and 4 tb of memory, hmmm!

EYPC 7742, yup

I’m not sure But things like that are not typically DB heavy

Good day, everyone! The extended downtime has started. We will keep you posted here and on EVE Status Twitter on the progress.

Tests went well. Thank you everyone for your patience. Tranquility is now back online accepting connections!

Please remember that tomorrow we will have another extended downtime with estimated duration of 30 minutes.

If you are having payment issues or did not receive PLEX you purchased, please contact customer support for assistance.

We may have had a CPU spike just after failing over… but its all good now

It is a MS SQL server, and its not fun in any way to hint or think about the license cost

Close! 64 cores is right (we have SMT enabled, so 128 HT cores) but it has 2 TB of memory

This is actually just a proof of concept… it’s a loaner box to see how we like it and how the AMD architecture will work for EVE (I love our vendors!). But I can’t imagine we would ever fill the other socket, at least not with another CPU like this.

we’ve dropped from 30% CPU to well under 10% - at this time of day things are typically calm - CPU is rarely balanced across cores due to the nature of how SQL works - it could using a single core for a call or a parallel plan that would hit multiple cores.

I can say that we’re seeing much more balanced NUMA nodes (groups of cores) which is one of the major things we were after here compared to our old setup

image1653×890 42.6 KB

We’ll do a devblog at a minimum and have had other people mention a stream… will look into it further!

CCP RAM and I did a presentation a few years ago at fanfest - it’s a mix of how we came to CCP and daily Operational tasks/tech talk at the end: https://www.youtube.com/watch?v=w8rGZCj6rgQ