It looks like I've cracked the case on where Global Crossing just is plain wrong, with the help of Bernard, who pointed me in the right direction.
I did some tcpdumps on the IPv6 session for Neighbor Discovery and Solicitation, which uses ICMPv6 and is crucial in the interaction of IPv6 hosts. And guess what ? They are even rate-limiting that !!! No wonder, we're constantly loosing the connection to their router.
Anyhow, I've dumped more data including the tcpdump to them and we'll see, what they say to that.
Saturday, March 28, 2009
Thursday, March 26, 2009
Mikrotik Queues broken AGAIN !! (1/3)
I discovered, that the queues in RouterOS can be escaped again. The particular setup, where I'm seeing this is a VPLS circuit, bridged to a lan-port. Queuing is established in the manner, as they suggest, by enabling firewalling for bridging and create the queue on the bridge.
So I send them a mail on the 22nd (4 days ago) about that and raised a ticket. Answer today is, they can't reproduce the problem. Guess I've got to figure that one out myself.
So I send them a mail on the 22nd (4 days ago) about that and raised a ticket. Answer today is, they can't reproduce the problem. Guess I've got to figure that one out myself.
INEX Meeting, GBLX, various.
The INEX meeting in Dublin was on. I gave a talk on IPv6 deployment. The video for that can be found here.
It also looks like in the likes of the issues with GBLX, we're probably going to kill that session and get IPv6 from KPN instead. Global Crossing engineers have no understanding for, that rate-limiting ICMP in IPv6 simply is a no no. Even though they are saying, that they only are limiting the ICMPv6 traffic targeted for their router. It clearly affects us badly. How bad ? Check this out:
The gap that looks ok from the 23rd to the 24th is, when they shut the filter down after much arguing. However on the 24th, after we confirmed it is the filter, that is causing the issue, they just bloody turned it on again. Disregarding that the service is useless to us that way.
It also looks like in the likes of the issues with GBLX, we're probably going to kill that session and get IPv6 from KPN instead. Global Crossing engineers have no understanding for, that rate-limiting ICMP in IPv6 simply is a no no. Even though they are saying, that they only are limiting the ICMPv6 traffic targeted for their router. It clearly affects us badly. How bad ? Check this out:
The gap that looks ok from the 23rd to the 24th is, when they shut the filter down after much arguing. However on the 24th, after we confirmed it is the filter, that is causing the issue, they just bloody turned it on again. Disregarding that the service is useless to us that way.
Wednesday, March 25, 2009
Adding 200 mbit/s from Galway to Dublin
It looks like the next couple of months will be very exciting.
We've recently peaked 86 mbit/s on our BT circuit to TeleCity in Dublin and the Smart circuit on the other end of town is running at around 23 mbit/s. That's an aggregated traffic volume of approx. 110 mbit/s at peak.
Now, we're going to shut the Dangan circuit down, maybe keep it as backup as it doesn't cost us much, but all the traffic on that circuit is going to be moved into our datacenter in Mervue. Also with only 14 mbit/s leeway left, more bandwidth was needed.
Smart is going to supply us with 100 mbit/s L2 (in addition to the BT circuit), which we're going to combine on the Cisco switches I've just bought for our network.
On top of that, we're going to get another 100 mbit/s L2 circuit into DEG in Dublin (another datacenter), where we are moving our INEX Lan#1 connection. That way, we'll be connected to INEX in two different datacenters and will be getting the optimum redundancy out of that.
That's a total of 400 mbit/s from Galway to Dublin now, where 300 mbit/s of that are in our datacenter in Mervue.
There were a lot of reasons for not increasing the circuit with BT:
- we wanted fiber into the building on a physical different path. Smart was not even allowed to bring the fiber in through the same ducting or elevator shaft. So matter of fact, we're bringing the fiber in through a cable-riser in the neighboring building and then through a complete different wall.
- we wanted the fiber on an entirely different path from Galway to Dublin. BT's fiber is along the irish railroads, while Smart uses ESB networks fiber along the pylons and high voltage lines.
- BT only installed a STM1 capable CPE, when they supplied us with the fiber initially. If we had opted just to increase that circuit, the CPE gear would have to be replaced resulting in downtime or us hauling the traffic across town to Dangan, which isn't really an option. The reason for establishing the DC in Mervue was, that we couldn't get decent wireless links to Dangan.
These are just a few of the reasons, why.
We've recently peaked 86 mbit/s on our BT circuit to TeleCity in Dublin and the Smart circuit on the other end of town is running at around 23 mbit/s. That's an aggregated traffic volume of approx. 110 mbit/s at peak.
Now, we're going to shut the Dangan circuit down, maybe keep it as backup as it doesn't cost us much, but all the traffic on that circuit is going to be moved into our datacenter in Mervue. Also with only 14 mbit/s leeway left, more bandwidth was needed.
Smart is going to supply us with 100 mbit/s L2 (in addition to the BT circuit), which we're going to combine on the Cisco switches I've just bought for our network.
On top of that, we're going to get another 100 mbit/s L2 circuit into DEG in Dublin (another datacenter), where we are moving our INEX Lan#1 connection. That way, we'll be connected to INEX in two different datacenters and will be getting the optimum redundancy out of that.
That's a total of 400 mbit/s from Galway to Dublin now, where 300 mbit/s of that are in our datacenter in Mervue.
There were a lot of reasons for not increasing the circuit with BT:
- we wanted fiber into the building on a physical different path. Smart was not even allowed to bring the fiber in through the same ducting or elevator shaft. So matter of fact, we're bringing the fiber in through a cable-riser in the neighboring building and then through a complete different wall.
- we wanted the fiber on an entirely different path from Galway to Dublin. BT's fiber is along the irish railroads, while Smart uses ESB networks fiber along the pylons and high voltage lines.
- BT only installed a STM1 capable CPE, when they supplied us with the fiber initially. If we had opted just to increase that circuit, the CPE gear would have to be replaced resulting in downtime or us hauling the traffic across town to Dangan, which isn't really an option. The reason for establishing the DC in Mervue was, that we couldn't get decent wireless links to Dangan.
These are just a few of the reasons, why.
Sunday, March 22, 2009
GlobalCrossing, IPv6 and Packet Loss. AGAIN !!
Remember December 16th ?
Well, here we are again. As of the March 18th, the packet loss, flapping BGP sessions, etc is back. I guess the whole game starts over again.
Well, here we are again. As of the March 18th, the packet loss, flapping BGP sessions, etc is back. I guess the whole game starts over again.
Wednesday, March 18, 2009
Server Racks for Mervue
eBay is good for many things. We bought 7 full height racks on eBay. With 5 KVM's, tons of power and network cabling and a a lot of shelfes fit in the racks, this definatly was a bargain.
We made a day trip out of it, rented two Ford Transit vans and collected the racks in Dun Laoghaire ourselfes today.
We made a day trip out of it, rented two Ford Transit vans and collected the racks in Dun Laoghaire ourselfes today.
Wednesday, March 4, 2009
CeBIT
After many years, I managed to get to CeBIT again.
With 25% less companies showing their products and 20% less visitors it actually turned out nicely.
It was bearable, no rush, no stress and yes, your feet still hurt at the end of the day.
The important tasks for myself was to find some WDM fiber gear, some gear for licensed microwave links and maybe somebody who manufactured CPEs in a quality acceptable for us.
It looks like I've succeeded in all those tasks. Happy days.
A few notes:
- AVM launched their lab firmware for the Fritz!Box Fon WLan 7270 with IPv6 (Yay !)
- TP-Link doesn't even know what IPv6 is, or practically denies the fact of their knowledge.
With 25% less companies showing their products and 20% less visitors it actually turned out nicely.
It was bearable, no rush, no stress and yes, your feet still hurt at the end of the day.
The important tasks for myself was to find some WDM fiber gear, some gear for licensed microwave links and maybe somebody who manufactured CPEs in a quality acceptable for us.
It looks like I've succeeded in all those tasks. Happy days.
A few notes:
- AVM launched their lab firmware for the Fritz!Box Fon WLan 7270 with IPv6 (Yay !)
- TP-Link doesn't even know what IPv6 is, or practically denies the fact of their knowledge.
Subscribe to:
Posts (Atom)