Founded in 2017, Australia-headquartered payment platform Limepay had a minimum viable product (MVP) and when in December 2020 the company raised $21 million in funding it decided it was time to improve its application.
When Limepay started out, it chose Amazon Web Services to build the first iteration of its platform. After the capital raise, the company started investing in a team with experienced IT professionals to start rebuilding and then scaling its product so it would be a solid and scalable one, Limepay CTO Andy Britz tells Computerworld Australia.
Limepay selects a cloud platform to rebuild a solid, scalable application
With a team of experienced engineers and IT professionals in place, and with financial support, Limepay went on to evaluate the cloud infrastructure provider options with a series of criteria that needed to be met by either the existing provider or a new one.
After evaluating the market, Limepay found that Google Cloud was a better fit for the company, especially from a data perspective in how the organisation wanted to work with data. “BigQuery was a big cornerstone as part of our selection in terms of the data side of things and also, like most companies today, we've gone with the kind of containerised architectures using Kubernetes and all the rest of it. And we felt that, from our perspective, Google was slightly stronger there,” Britz says.
Limepay
Andy Britz, CTO at Limepay
Another key point was that Google, Britz says, brought together its various components better than AWS did at the time. “AWS has an incredible wealth of offerings, but they feel a little bit more disconnected than what we got from Google.”
After choosing to change from AWS to Google Cloud, Limepay decided to rebuild its entire application with its new preferred cloud provider in mind. The rebuild took less than a year, and the new app has been running on the new cloud platform for nine months.
How Limepay worked with two cloud providers during the transition
The transition did not happen in a day. Limepay ran both applications in the two different clouds for a period of time. This was followed by a period where both apps shared some services before the final switch.
“It was a gradual move from the legacy code base to the new one. As we brought new features online, we migrated those across to the new platform. So, for the better part of 12 months, we ran a dual cloud strategy and now we're in that phase where we still have a small footprint in AWS but that will probably be shut down in the next three months,” Britz says.
Limepay built a mostly cloud-agnostic platform. The transactional side of the application is agnostic, Britz says, but they still use MongoDB Atlas as the primary database. “That’s largely because of legacy reasons. It was easier to maintain that database, and whilst we did strongly consider moving to a relational model, there wasn't enough of an incentive for us to do that, especially since our choice of software models was to use Scala, which is well typed and functional. We could get around some of the trade-offs in terms of schemas and things like that by moving that into the application space. So, we made the early decision to stick with MongoDB, but everything does run on the Google Cloud,” he says.
Limepay selected Scala as they wanted a strongly typed functional language, especially when working in a transactional domain that demands a high degree of correctness. “Although our choice of cloud provider did not directly influence our selection of Scala, Google’s long history of hosting JVM-based applications did factor into our cloud selection criteria,” Britz says.
Finding Google Cloud-experienced devops staff proves to be a challenge
Moving to a cloud provider that is not the market leader can mean it is difficult to find a large group of tech specialists for that other cloud provider. And that was the main obstacle Limepay faced, Britz says, as finding a great group of devops professionals was a challenge.
The IT team had the option to learn on the go, which, Britz says, could have been achieved but they decided that to save time by having someone with experience who could help also training the existing team was a better solution and it would also avoid making mistakes along the way. Thus, Limepay hired Tom Murphy, who was experienced with Google Cloud, to help lay the base architecture from the networking setup.
Another decision that helped support the talent challenge was automating the infrastructure, which Murphy also helped make sure that had been set up, so ultimately Limepay could rebuild environments, make changes that are tracked, and ensure everything has versions. “That was probably the biggest challenge. A lot of people automate parts of their infrastructure but not all their infrastructure. And we were very strict on deciding nothing that we deploy will be deployed through any kind of UI or anything like that,” Britz says.
Another decision was to avoid what Britz calls the “expert syndrome”, meaning if they were to lose Murphy then no one would have a clue what to do. So Britz made sure was that everyone in the team to some degree was across how the infrastructure works, and he nominated a few experts so the team was comfortable in being able to continue to do the work if anyone were to depart the team.
Limepay also had the support of Google Cloud representatives who supported the team along the way by validating ideas, as well as with training. “For instance, on our data side of things, we also adopted their visualisation tool, Looker, and they made provision for training for that. So, our analysts got trained on how to use the tool optimally and set things up correctly from the start. They worked quite hard with us on that; they allocated quite a number of hours to work with us through those models as an example,” Britz says.
Setting up a safe and compliant network environment
One change while building the new application was in how Limepay worked, as it must comply with several regulations to be able to operate. Limepay has strict rules of network segmentation and constraints in how it sets up its security.
The first thing the IT team focused was on the design of how the networks had to be set up to enable development, testing, staging, and production environments to work, which was where a lot of time was spent to make sure the system was compliant.
The team then focused on setting up the Kubernetes clusters, which brought new challenges. This is because Kubernetes “brings a whole different way of doing things in terms of how you schedule tasks and all the rest of that,” Britz says. One challenge was around how they had cron-job-type tasks running: Moving them into the “Kubernetes way of doing things” involved some work and was also a learning curve in terms of engineering.
BigQuery saves Limepay from manual data analysis
One key platform for Limepay is Google’s BigQuery data analytics. Britz says that before moving to the new cloud, the organisation wasn’t using any data analytics tool, and BigQuery’s native integration with Google Cloud was one reason Limepay chose Google Cloud. (BigQuery also has integrations for AWS.)
“We were effectively doing most of our reporting at that time off the main transactional data system, which was MongoDB. We’ve put a lot of work into our data pipelines to move our transactional data into BigQuery, and now we have about a couple of seconds of delay. So the data in BigQuery is only a few seconds out of date from the transactional system. Then we used Looker as a tool to define a semantic model on top of that, and we use that for all of our dashboarding and visualisations internally at the moment,” says Britz.
Limepay has also plans to start to embed some of the Looker dashboard and analytical views into its core application.