Data Services News: Tying Two Data Centers Together Using Global Load Balancers
Operating a large data center like the IRIS DMC clearly has many challenges. But there are “core” challenges that represent persistent administrative complexities, including scaling to non-linear growth across both the archiving and the distribution of data. For example, the volume of data distribution is currently exceeding 1.2Pb annually, and the persistent level of connections to service.iris.edu exceeds 1 million per hour. This article addresses one new technology that insures that IRIS Data Services (the primary FDSN archive) can meet the demands of the coming years, called Global Server Load Balancing (GSLB).
IRIS Data Services operates two geographically separated data centers, preparing us for disaster recovery functionality for important applications like webservices, in addition to enabling more intelligent direction of internet traffic for optimal site selection. The primary center is located in Seattle at the IRIS DMC, and the Auxiliary Data Center (ADC) has been located at the Lawrence Livermore National Lab (LLNL) in Livermore, California since 2013. In December of 2017, we incorporated a Global Server Load Balancing appliance from A10 networks to not only address disaster recovery functionality, but to help us with intelligent traffic direction for optimal site selection. These appliances (there are two at each site for redundancy) operate in DNS Server mode, where all services that are managed by our GSLB have a DNS record in the primary domain and resolved by the delivery controller so that traffic can be directed to either data center based on load, planned maintenance, or other needs. Currently, the data center selection for our external traffic is set to favor the ADC site by a ratio of 5:1, to address the increased resource load currently spiking in Seattle related to MUSTANG and similar internal demand.
These appliances are game changers, because they remain in constant use, validating that all endpoints of services are resilient, synchronized, and reliably answering the question “Can a user get the same information (data, service.iris.edu response, etc.) output from either domain when one side is down, transparently and reliably?”.
After months of continual operation now, we are very happy with the interaction between the two data centers.
by Rick Benson (IRIS-DMC) and Inge Watson (IRIS DMC)