CCP_Explorer

CCP_Explorer



07 Mar

Comment

I’ve heard some unconfirmed reports that “Total Play Telecomunicaciones SA de CV” did some upgrades on their infrastructure recently to (better) support IPv6. Maybe there was some fallout there in supporting IPv4 properly, or there are IPv6 <=> IPv4 translation issues. Perhaps also check with them on that angle.

tranquility.servers.eveonline.com (TCP, port 26000) is at 172.65.201.188 – only on IPv4. That is the address you get for TQ?

Comment

After Virgin Media made a fix in the early afternoon on 3 Mar (between 13:30 - 13:45 UTC) then all was good over the weekend until last night when there were small disconnect spikes at 01:01 and 03:25. Since then all been quiet.

I replied to the email thread we had with Virgin Media and Liberty Global to let them know of this latest development.

But: If this continues then you, as customers of Virgin Media, need to open cases with them. Then please reference your previous ticket(s), all the details here in this forum thread, as well as the details in ...

Read more

06 Mar

Comment

Thanks; and if you could also do me a favour as well.

Please find the support ticket you filed with us and make sure it includes

  • IP address
  • ISP name
  • Region location
  • City

as well as

tracert tranquility.servers.eveonline.com

You probably/hopefully submitted the network diagnostic results, but I would also like to see what the standard trace route tool outputs.

Please include this in your ticket and kindly then post the ticket # here.

Finally, if you have telnet on your machine (or can install it), what is the output from

telnet ...

Read more
Comment

Everyone that has raised a ticket with us from Mexico is using “Total Play Telecomunicaciones SA de CV” as their ISP, as far as I can tell from the data we have gathered so far.

Is that the largest ISP in Mexico? Anything that might explain why EVE players in Mexico might only be using “Total Play Telecomunicaciones SA de CV” as their ISP, or why only those players are affected and not players using other ISPs in Mexico?

Is “Total Play Telecomunicaciones SA de CV” partnering with other ISPs in other countries in Latin America? I see the complaints from other Latin American countries are bit more diverse when it comes to ISPs.

Is “Total Play Telecomunicaciones SA de CV” your ISP? If so, please open a ticket with them and ask them if they are blocking, scanning, filtering, proxying, or manipulating your network traffic in any manner. This would include if there are transparent proxies. In particular ask them to make sure that traffic to

Read more
Comment

We’ve had a few reports recently about this particular issue, but only from EVE players in Latin America – mostly Mexico and Chile. Based on metrics (particularly daily active users from Chile) then this started on 24 February.

The errors are puzzling. Based on the logs we have seen so far then the connection to the Server isn’t blocked and it isn’t terminated by the Server or any network equipment. Rather there is this in the logs Invalid answer from server GetServerStatus when the Client tries to authenticate to the Server.

Do you see that error in your Client logs?

We suspect that somewhere there is possibly some network traffic inspection (filtering or proxy) since we are seeing CORS header errors in the Launcher as well, indicating that the headers have been stripped by some middle layer, but primarily that mangled reply from the game Server to the Client that the Client is unable to decipher. There are sometimes also other network...

Read more

05 Mar

Comment

@Adelrad, you are not experiencing disconnect issues but rather that you can’t connect at all? And in the Client logs there is an “Invalid answer from server GetServerStatus” error?


04 Mar

Comment

This thread is only about the disconnect issues that were affecting Virgin Media UK customers. For your issue, which is of a different nature and with a different ISP, please create a technical support request.

Comment

UK status since 1 Mar:

image1140×541 68.4 KB

I’ll update here and on Twitter on a few days with longer-term data, but this case is closed.

Comment

You are in Mexico or Chile? If, please file a support ticket.


03 Mar

Comment

Could be.

We explicitly suggested to VM on Monday evening that they “try a different egress point to reach 172.65.192.0/20”.

Prior, early in the morning on that same day, we had suggested they could “perhaps change that for 172.65.201.188 (or for Cloudflare in general, AS13335) and route directly within the UK to Cloudflare? Perhaps even initially only as a test to see if a new route makes a difference.”

BTW, the fault was that VM’s systems were trying to establish this direct peering (that they ended up with) but their systems were failing because of prefix limits being breached.

Comment
Read more
Comment

I’m all ears (not literally this time) what that other alternative should have been.

The internet isn’t a single unified entity; it’s a network of networks glued together with BGP and we here at CCP only control our own network.

Comment

This is looking good since Virgin Media UK’s fix this afternoon:

image1143×535 60.2 KB

Comment

@Lacrimosa: Referring you to my two posts right above, please see

  • Right, now I’m interested in hearing if you are all getting new routes?
  • Using one of the trace routes from above for some quick analysis

then please note that Virgin Media UK literally fixed the issue themselves by literally fixing

“Issue looks to be due to flapping BGP between Cloudfare and Virgin Media due to prefix limit breach.”

on literally their own equipment, thereby literally fixing/re-establishing their own direct peering with Cloudflare. None of this was or is under CCP’s control or under the control of our internal network partner Advania or our external network partner Cloudflare or any of our network carriers or direct network peers.

...
Comment

I don’t have the details to be able to answer this question – only Virgin Media would be able to tell what the prefix limit was before and after on their equipment, what alerts there are on the limit being breached and their direct peering with Cloudflare therefore being down, how that limit compares to the overall size of the internet’s BGP table and the number of their direct network peers, etc.

Comment

Last 10 days; now we wait to see if this graph stabilizes:

image1584×540 110 KB

Comment

Hmmm… but your trace route to TQ has changed to the direct peering? I have to admit, I’m slightly concerned.

Comment

Using one of the trace routes from above for some quick analysis and details to you, patient EVE players:

3 12 ms 11 ms 10 ms bagu-core-2a-ae70-650.network.virginmedia.net [213.106.214.185]
4 * * * Request timed out.
5 12 ms 16 ms 13 ms tcma-ic-2-ae9-0.network.virginmedia.net [62.253.174.178]
6 22 ms 42 ms 37 ms 162.158.32.254
7 16 ms 16 ms 18 ms 162.158.32.11
8 18 ms 18 ms 14 ms 172.65.201.188

You will note that there is no more Telia in between Virgin Media and Cloudflare, no more Liberty Global, and no more Amsterdam.

Rather in hops 5 an...

Read more
Comment

Right, now I’m interested in hearing if you are all getting new routes? Virgin Media just told us “Issue looks to be due to flapping BGP between Cloudfare and Virgin Media due to prefix limit breach.”

You should now be getting routes that go directly to the nearest Cloudflare PoP, of which there are 3 in the UK: Edinburgh, Manchester, London – so depending on if you are in the north, south or center of the island then you should get the different PoPs and your route should stay that way during your EVE play session.

EDIT: If Virgin Media is only peering with Cloudflare in one of those three locations then you will all be routed through that location. It looks like Virgin Media is peering with Cloudflare in Manchester.

Comment

Actually, @Alaska, can you also post in the ticket the regular trace route, please; for comparison?