Paytm, like any e-commerce company, has lots of data to process, moving data across different systems of varying complexity.

A traditional enterprise would have various IT groups handling the process of connecting new and old systems. They meet, plan, discuss, plan and often build complex middleware to translate data between systems. With Paytm’s mix of enterprise IT and devops engineering, we can simplify difficult issues with better tools (and maybe smart people). A typical business case might require historical analytics merged across many different business verticals. Generally, the owner of the business case has limited knowledge of what is involved in moving data around. They may not know about the complex web of legacy networks, old firewalls, overlapping subnets, varying security policies and processes. In many cases, we get the job done with a few tech tricks and better communications which builds bridges between teams.

Interactive Chat

Communication and coordinating between teams is critical, but can sometimes be difficult. Simple business tools such as Email eventually become wastelands of documents, sometimes leaving people buried. If you want to avoid spending all day in meetings, nothing comes close to real-time chat. Services like hangouts, teamviewer, and slack allow teams in different parts of the world to quickly share information, pictures or links. Why is this important? 1:1 communications in real time is a very efficient way to operate. Traditional offices promote teamwork, interactivity, and meetings. This isn’t always possible for systems and network operations. For example, when Paytm set up VPNs between server clusters and public clouds, a quick text, group chat or shared desktop will clear a dozen hurdles that might take weeks if it was managed by email only or in person. Once everyone is on the “cool” chat system of the day, the use of chatops bots can get quite interesting. We have chatops tools for reporting task updates, monitoring alerts, code commits, build status, and much more. Technically, you could use text chat to build all your servers and run your applications. However, just because you can, doesn’t mean you should…

Tech Solutions and Duct Tape

When the information technology team needs to bridge data from one sytstem to another, there are often many hurdles to overcome. In many cases, you move between systems that were not intended for data transit.

If you want to allow systems on one network to talk to another, but you don’t have the ability to control routing policy, set up a simple haproxy config. A generic backend config for haproxy would look like this:

listen 10.0.0.100:2222mode tcp

mode tcp

option tcplogbalance

balance leastconnserver server02 192.168.1.100:22 check

server server02 192.168.1.100:22 check

Testing if services are working might look like this:

> nc -v 10.0.0.100 2222

Connection to 10.0.0.100 2222 port [tcp/ssh] succeeded!

SSH-2.0-OpenSSH_5.3

When you move large amounts of data, you may need to consider traffic shaping. This can be a complex process, but worth the effort when you need to optimize one type of traffic over another. Playing nice with other teams is also key. That said, i have used a pair of HAProxy systems in a chain to get around problems, until they can be properly solved.

Moving data from one place to another usually comes down to specific technical issues. A network on one side can’t be changed to route directly to a new VPN, or subnet. Instead, setup HAProxy and allow almost any TCP session to move between isolated networks or preferred paths. Nginx and other HTTP servers can proxy and change data within some protocols as well. This avoids frustrating client proxy or ssh tunnels between different hosts. I’ve used haproxy for basic HTTP/HTTPS, or more interesting use cases between MySQL hosts, DNS, Elasticsearch, and RabbitMQ. Some protocols, however, don’t work when IP addresses change between source and destination. Kafka was one example (this may have changed in recent releases). To expose an admin console for a tool like cobbler, you can simply add the following, with the private IP of your cobbler server:

   location /cobbler_web {

proxy_pass http://10.0.0.2/cobbler_web;

}

location /cblr {

proxy_pass http://10.0.0.2/cblr;

}

We try every decent tool. Technology such as Docker is changing how accessible the tools are for a broader range of users. Check out the Docker Hub for examples like HAProxy or Nginx. Feel free to reach out to me if you have questions or comments. I can be reached on

Feel free to reach out to me if you have questions or comments. I can be reached on Twitter.