EVE Online Dev Tracker

Yes (mostly) and no. The routing even if inefficient doesn’t necessarily have to be the issue. But as many players in this thread have reported, using a VPN from the UK to the UK is bypassing the fault. so it could be something with the route or on the route.

Yep, VPN-ing, even from the UK to the UK is bypassing the fault. We don’t know at this time if the fault is a routing issue, some filtering issue, BGP flapping, a bad port on a router somewhere, a device in VM’s network that runs out of memory and restarts, …

We are monitoring that different issue, which is mostly manifesting in Chile but also a few EVE players in Mexico have reported. In those cases the connection appears to be outright blocked, whereas as the issue in this thread is that Virgin Media UK EVE players are able to connect and play but are then synchronously disconnected. Please send us a ticket about your issue. We have a code change in the pipes to get better logging of what is happen...

Hey @Induce_Deadline: Can you do me a favour? Create a support ticket, reference the ongoing VM issues and include a trace route so that we have a BT comparison? Please name drop me and GM M...

Virgin Media appears to be routing the traffic on their internal backbone (or that of Liberty Global, who owns VM) to Amsterdam to or through an ISP in the Netherlands called VodafoneZiggo (also owned by Liberty Global). From there the traffic is handed off to Telia in Amsterdam who in turn sends it to Cloudflare in Amsterdam since Telia peers directly with them there. So Telia and Cloudflare are doing the correct thing given where the traffic is handed to them. But Telia and Cloudflare peer directly ...

End of day update:

We have been engaged with Cloudflare today looking at this issue, replying to Cloudflare’s requests for information. We have declared this a “P1 - Urgent” case, the highest priority level we have used since May 2020.

We have also been sending information to Virgin Media but we did not receive any replies today.

Removed some offtopic posts

Not really:

image1143×531 69 KB

Not according to our metrics or technical support. There was some noise in the metrics for German players overnight on the 23rd and again this morning (as in between 1 and 5 o’clock UTC) but fairly clean apart from that and nothing like the regular spikes at Virgin Media.

Exactly this; we see this clearly in metrics that the disconnects are synchronous and in support tickets that only Virgin Media users are affected. I’m hoping that timing of these disconnect spikes will lead Virgin Media to find the issue.

We opened a ticket with Cloudflare today to get more eyes on this issue. We haven’t had an update from Virgin Media / Liberty Global since this morning, when they told us they couldn’t see any issues. I’m trusting I should interpret that, that they couldn’t yet see anything wrong and would continue to investigate.

If they replace your hardware and everything starts working again, I would love to hear about it.

Yes, we are also working on issues in America, mostly Chile and Mexico. The symptoms there are very different since affected players there are not able to connect at all.

We have been able to reproduce this from one VPN point in Mexico. Based on the analysis so far of that failure then the response from TQ during the initial handshake is all garbled and so we suspect some sort of bad filtering.

Just a quick confirmation that we’ve had in the past issues with older Virgin Media hubs. Those issues should be known to Virgin Media and be resolved by now; and also would not explain these synchronous disconnects we see from the UK. Also a quick confirmation that customers of other ISPs in the UK are not complaining to us, only Virgin Media UK customers.

Hey Capsuleers – we have an update with the DBS situation and appreciate your patience as we went through our investigation.

Many players noticed drastic changes to the Bounty Risk Modifiers in their solar systems beginning on February 16th. We immediately investigated this issue and concluded that, while the external system doing the Bounty Risk Modifier calculations was still receiving the correct data, it was not properly relaying that information back to Tranquility.

We are acutely aware of how the Bounty Risk Modifier in the solar system dictates player decisions when it comes to choosing a suitable location, and the impact that erroneous information has. Due to this outage, we will be restoring the Bounty Risk Modifier for every solar system. We will be looking at the Bounty Risk Modifier from the date of the outage and the date of the restoration for each solar system, and restore the BRM to the higher of those two values.

The restoration will not be do...

Indeed; I find it a bit curious that Virgin Media doesn’t peer directly with Cloudflare in the UK (as I know other UK-based ISPs do) but there could be reasons. What I do find curious is to bounce the traffic to Amsterdam and hand it there over to Telia, instead of handing it over to Telia in the UK (as Telia peers with Cloudflare both in the UK and in the Netherlands).

Per my reply above:

While it is very curious to send all-UK network traffic to Amsterdam then it is not necessarily the root cause of the issue. But to even begin trying to diagnose a cryptic issue...

EVE Online is very frugal when it comes to bandwidth requirements but can be sensitive to packet loss, latency spikes, over-capacity spikes, and BGP flapping. It needs a stable TCP/IP connection, which is why we have always invested in good networking equipment and partner with companies such as Cloudflare.

EDIT: Virgin Media UK did provide such a stable TCP/IP connection until 22 o’clock on 23 February

...

Disclaimer: While I often use Linux, I do not use it for gaming. I have not investigated what I’m about to say.

CCP has been trying to break apart our monolith for a while. Skill plans were one of the first things to get this treatment. There is a devblog over here that has some information that may...