QA Testing Blog | Global App Testing

How GAT helped Carry1st improve checkout completion by 12%

Written by Ankur Mishra | November 2023

Carry1st is an African gaming and payments platform which recently raised $27M led by Andreseen Horowitz. Below, we spoke to Ankur Mishra, a Senior Payments Manager at Carry1st, who helps to ensure their payments work seamlessly for consumers and gaming companies alike.  

Interested in learning about how we could help you? Talk to a member of our team.

The company

Carry1st gets between two enormous markets. The first is African consumers wanting to pay for their favourite games with local payment methods which are convenient to them. The second is gaming companies who want to reach African consumers but are not set up to cater for the huge variety of payment options.

Enter Carry1st. “We’re integrated with all the major gaming partners; we’re acting as a payment gateway aggregator as well as an online shop," explained Ankur.

Founded in 2018, Carry1st is now available in 7 African countries following a successful Series B. They offer “a hyper-localized experience. There is no need to buy with a card if you don’t want to. We’re able to offer over 120 methods of payment to our users.” 

That includes things like bank transfer, crypto, and the thriving number of e-wallets which are popular in Africa, such as Nigeria’s OPay wallet.

The challenges 

We asked Ankur about his top three challenges in QA at Carry1st. He told us the following:

  • The complexity of testing 120 payment methods
  • Testing a hyperlocalized product
  • Driving low levels of friction required to encourage purchases

Testing 120 different payment methods 

Payments are fundamentally very difficult to test, argued Ankur. "We need a lot of simulators, work around the staging environments, integration testing… then there’s checkouts and driving down the friction on them. We test the initialized payment, the requested responses. We do some unit UA and sanity testing in the lower environment. "

"We have some limitations when we do end to end testing. We struggle to coordinate international payments… we need a method like bank OPay wallet which is in Nigeria. And there’s often no sandbox or simulator available. In lieu of that we have to do some very basic test cases and based on that we make a call on whether we go into production or not. "

"The integration aspect is challenging because the customer gets redirected to the bank infrastructure, meaning we lose the control. And then the security systems designed to prevent spam and fraud prevent automated scripts from paying. So we need real testers on real devices."

Hyperlocalization across 7 countries

“We’re a remote company,” said Ankur. “But even we can’t get testers in all the locations we need.” 

 
The difficulty of getting QA right across such a diverse set of countries is partly the regulatory burden associated with each one. But it's deeper than that. Asked what Ankur meant by hyper-localization, he said:
 
"It’s also the localized content and UX. For example if you’re using Carry1st in Nigeria, you need to see the list of the 40 banks that support transfer against a list of all the banks in Nigeria. You can check yours in a dropdown list." 

"We need to be able to test that the information is correct, and to deploy localized changes which are deeper than translation across individual territories in our markets." 
 
 

The level of friction required for revenue capture

“The business drivers for high quality also go beyond ensuring that the payments work,” said Ankur. 

“If I’m playing a game and I want to make an in-app purchase, my use case is often immediate. That means the purchase won’t happen if I have to wait ten – or even two – minutes. One of the big value adds we can provide to our gaming partners is capturing that revenue which would otherwise be lost."

But with 120 methods of payments, ensuring that everything is frictionless requires constant vigilance. “We’re trying to reduce that time to purchase as much as we can, and trying to reduce processing time as well. The user experience should be that a user can do in-application payments without friction and resume their game as soon as possible."

 

 

The solution

Global App Testing delivered a bespoke solution to Carry1st which was tailored to their needs and requirements.

Real testers, real devices, real payments

After testing as much as they can with automated scripts, Ankur is grateful to use real localized device testing to ensure that the product is working in practice. 

"We run a set of test cases in production using real money, real cards, a real bank account a real e-wallet and the real movement of funds. It gives us something that all the sandbox testing can’t – local functional testing on payments in a real environment where we can’t access internal testers." 

End-to-end checkout testing across releases and the live product 

"We do end to end testing with Global App Testing in production of all the new payments that we introduce," said Ankur. "Plus we use GAT to investigate anywhere we think that the performance isn't where it should be to identify bugs which might be impacting the metrics." 

"The end-to-end test cases for all payment flows are extremely value as they give us a full picture of the entire UX, in practice, anywhere in the world." 

 

The results

A 12% increase in checkout completions with some payment types 

“There’s been a few extremely valuable defects that Global App Testing has found. It’s been acknowledged by the entire Carry1st leadership”, said Ankur.

“There were a couple of incidents where payments appeared to be cancelled by users but they were actually issues with the iframe loading. From the internal side they looked like genuine user cancellations. Because of the specific pattern of environments these bugs occurred in, the failure rate was low enough that it did not flag as suspicious but high enough that it incurred significant cost.

“The success rate on that payment went from 78% to 90%. There would have been at least $300,000 of daily volume going through that channel.” 

A richer understanding of the whole system

“Global App Testing helped me a lot in terms of understanding the end-to-end of the payment method.

"Attaching the videos and recordings of the full session is so valuable. It’s become much easier for me to understand how this payment system works – and for our developers to find where the bugs are – because there’s media with structured environment and location details of every test case."

"I'm able to control the payment operations. Whenever I see issues, I can quickly understand the cause.:

Carry1st lines up their biggest partnership to date

We asked Ankur about Carry1st’s growth plans for 2024 and to our surprise – Ankur responded that he couldn’t tell us.

“We’re planning to extend our business in two more countries this year; but we have new partners coming in and some exciting stuff happening there. This year we’ve integrated into four additional [gaming] partners…. At the end of the year we’ll have two more.”