Zerodha: The Unicorn in Indian Trading Sector
Major equity markets across the globe, including in India, have seen a rally since the crash in March. This seems to have attracted a lot of new investors to the market. New customer additions have now almost doubled per month than before COVID-19 hit, Nithin Kamath, co-founder and CEO of Zerodha, the country’s largest broking firm, told in an interview.
Zerodha is the leading discount brokerage in India, providing online trading services in stock, commodities, mutual funds, bonds, and currency, having around 30 Lakh customers now, of which active customers are around 18 lakh to 19 lakh. In terms of trading volumes — it in a number of trades — averaging 50 to 60 lakh trades daily.
Launched in 2010, the company has huge clients placing millions of trades each day. Zerodha was one of the first discount brokerages in India. It charges 20 rupees (US$0.29) per trade or 0.01% of the trade’s value, whichever is lower, and does not take brokering fees. In 2018, the company became the largest retail stock brokerage in India, according to the National Stock Exchange of India.
During initial days, while Zerodha handles 2 to 3 million trades a day on average, this figure can easily double on the back of breaking financial news. Because Zerodha leased on-premises servers at a hosted data center, the company experienced issues scaling its architecture to meet trading highs and lows, and, as a result, it had to overprovision its IT. Kailash Nadh, chief technology officer at Zerodha, says, “Most of our capacity could go unused for days or weeks during non-peak periods. It was like a very expensive insurance policy.”
To reduce the cost and increase efficiency, Zerodha requires infrastructure that spans a public cloud (AWS for most in-house applications) and physical machines in multiple data centers, with specific regulatory and technical requirements for capital market connectivity via leased lines and adapters from various stock exchanges.
That complexity — along with a heavily regulated technology stack and end-user applications and internal systems with disparate external dependencies- led the company to embrace cloud native technologies.
Storing Files on AWS S3->
When Zerodha learned that AWS was expanding its Asia-Pacific Region to Mumbai, it gave the company the opportunity to use the AWS Cloud. In 2016, Zerodha moved its IT infrastructure from on-premises to the Amazon Web Services (AWS) Cloud to scale to its fluctuating trading volumes and comply with India financial regulations that limit data sharing across geographic locations. At this point, Zerodha was using Amazon Simple Storage Service (Amazon S3) to store large documents. Says Nadh, “We were very satisfied with Amazon S3, which was eight times more cost-effective than our on-premises storage. Furthermore, the relationship we had built with AWS around Amazon S3 gave us a lot of confidence to carry out a full migration.”
Zerodha managed its migration internally — building applications for its website, mobile app, and backend processing by using a modular design. This modularity was also what made it easier to migrate to the AWS Cloud. “Because our software is stateless [where data travels independently of other data], it was easy to run on an internet-based cloud environment,” says Nadh. In total, Zerodha migrated all its IT onto the AWS Cloud except for a few on-premises servers used for live feeds from India’s stock exchanges. “If the Indian regulations had allowed us to, we would have migrated those servers too,” adds Nadh.
Auto Scaling Using AWS EC2->
Apart from Zerodha’s website and mobile app, the company’s business-critical trading and post-trading platforms are also running on the AWS Cloud. Amazon Elastic Compute Cloud Auto Scaling (Amazon EC2 Auto Scaling) dynamically scales up and down the number of Amazon Elastic Compute Cloud (Amazon EC2) instances supporting the platforms, based on parameters such as central processing unit (CPU) utilization.
Approximately 500,000+ customers log on to Zerodha concurrently when the Indian Stock Exchange opens in the morning. Currently, Amazon EC2 Auto Scaling ramps up the number of Amazon EC2 instances supporting the trading platform during this period. The number of supporting instances is automatically scaled back down when trading traffic decreases.
Post Trading Process using AWS Lambda->
Once the stock exchange closes, Zerodha runs all accounting and settlement processes on its post-trading platform. This involves checking and updating files for about 500,000 customers overnight. “The work requires processing more than 10 billion data points and presenting the information back to customers, as well as financial authorities, in specific formats,” says Nadh. “At the same time, we’re updating analytical reports for our customers, which show their trading performance over weeks, months, and years. That requires recomputing their historical data with that day’s results.”
Before using AWS, Zerodha used a proprietary system designed for post-trade processing. The system, which processed data sequentially, took around 10 hours to complete the job. This is now done using AWS Lambda, which lets Zerodha run its software without provisioning or managing servers and completes processing tasks in parallel. “We’ve reduced post-trade processing times from between 6 and10 hours to 2 or 3 minutes with AWS Lambda,” says Nadh.
Overall Cost reduction->
The company has reduced the costs of its IT by 50 percent by migrating to AWS. It no longer has to over-provision because it can align processing power with trading activity. Zerodha has also cut the time and expense of managing its IT because much of the AWS infrastructure comes as a managed service. The administration required is done much faster. Says Nadh, “It could take two days to install a firewall at the on-premises site, but now it takes 2 minutes with AWS.”
Increased availability ->
The company has reduced network latency by 65 percent — from 200 milliseconds to 70 milliseconds (That’s huge difference). Furthermore, it has increased the availability of its services, reducing the risk of interruptions to trading while the stock exchange is open.
Using DevOps tools for greater efficiency->
Prometheus and Grafana ->
“Prometheus gave us powerful monitoring for critical, low-latency financial systems. It helped us aggregate and monitor metrics infra-wide. The large number of existing exporters and the ease of writing custom exporters enabled us to attain wide coverage in a short period of time.”
Kubernetes ->
Additionally, Zerodha began moving its services from VMs to containers, and gradually to Kubernetes in 2020. Because all of its apps had already been developed with a service-oriented architecture and 12-factor approach, the migration was straightforward. The infrastructure team began by creating CI pipelines with GitLab as the company has a well-defined process of pushing changes to production with its CI/CD process. With a focus on infrastructure-as-code practices, Zerodha uses a mix of Terraform, Packer, and eksctl to create its Kubernetes infrastructure on AWS and hosts container artifacts on an internal registry powered by AWS (ECR).
“Kubernetes has helped us standardize the deployment process of applications built on many different kinds of stacks across teams.” Quoted Zerodha’s CEO Kailash Nadh
Using AWS Infrastructure, Zerodha maintains around 99.96% Uptime that too with high efficiency.
This Shows the potential Cloud Computing has and AWS is dominating the industry by its low cost, highly efficient and scalable infrastructure.
You can connect me on my LinkedIn
Thanks To Vimal Daga Sir for encouraging to research on industry used cases.
