Beyond delivery: moving to the investment narrative
Evolving your maturity level
In addition to acing delivery (Chapter 1-2), there’s lots of advice available on how you can evolve your maturity level. We’ve listed them below, and they will form the basis for the second half of this playbook.
Here’s what we’ll cover in chapter 3.
Chapter 3 | Moving to the investment narrative
Identifying your ROI. How to calculate your ROI and describe a value narrative for your business
Building a data picture. How do you understand the impact you’ve had? How do you experience local users’ experiences?
Turning stakeholders into clients. Here’s how you begin to transform your role in the business.
How is the investment narrative different
After delivery, the second narrative is the investment narrative. The “investment” narrative says, “we invest more in our local offering by investing more in localization.”
Localization is a
Activity differentiated by depths & standards
Localization acts as
A barrier to releases
An investment in effectiveness
The L10n team is a
You are measured by
Expense & speed
Successful teams will
Run workflows better
Identify opportunities better
“Better quality” means
Fewer bugs or errors
Improvement in local metrics
You need to focus on….
Solid core materials
Automation & machine translation
Scope & suppliers
Prioritizing localization activities
Building value narratives
Opportunity-based quality insight
From the interviews: watch Francesca from Pinterest talk about the crucial distinction between that of a service provider and strategic partner
Identifying & communicating ROI
Identifying & communicating ROI is the easiest way to shift the dial in terms of how your team is perceived. But calculating ROI is much harder in localization – which is why our interviewees had a lot of advice for how to think about it.
What people say about ROI
The formula for ROI is
ROI = added value of l10n / costs of l10n
The formula seems simple, but it becomes much harder when we get to the metrics and costs involved in localization, on the next page.
A complicating factor is that you don’t “own” these metrics, you’re merely inferring an effect on these metrics. That would imply measuring the metrics before and after a localization activity, which is tough – especially if you’re localizing from the outset.
Added value metric examples
MAUs (monthly average users)
User session time
Reviews per platform
Cost component examples
Initial i18n investment
Other analyst costs
Other labor costs
Labor borrowed from other teams
It is not possible in a small l10n team to calculate all those numbers. For l10n teams with fewer resources, l10n relies often on a fuzzy sense of their value and a corresponding mistrust in their numbers.
Some of the l10n professionals we have spoken to off-the-record have described a struggle to articulate the ROI: they struggle to calculate it first of all; and then they struggle to generate credibility that their numbers are accurate.
Here’s what our interviewees had to say. 👇
ROI: the advice from our interviewees
1. Use analysts for the best results
Where possible, you should use analysts to do the work for you.
Francesca, Head of Internationalization @ Pinterest said, “I’d work with our team of [business] analysts and ensure that they had analysts dedicated to international [users]”.
Localization Operations Lead at Shopify said: “Return on investment is elusive. Everybody says there’s a formula. But at [former employer & HR management platform] Ceridian, I asked a colleague who calculates ROI for customers for a living to help.”
A HubSpot localization manager inanother podcastreferred to the use of analysts, too: “HubSpot Research is doing a lot of really original, data-based research…” he also describes the findings. “We’ve found that for certain locales, we’ve found that native-language content vastly outperforms localized content.”
Having somebody outside of the l10n department marking your effectiveness is also best practice in getting other teams to buy into the ROI figure. Not everybody has access to analysts, but you should at the least try to influence your data operations to create an international dashboard.
2. Attune your value frameworks to your organization
“You need to talk about [ROI] in your organizational language,” argued Oleksandr, the Localization Operations Lead @ Shopify.
“Let’s say [your employer is] product- and technology-obsessed. Then, you have to talk in those terms… create a roadmap. Talk about short- and long-term parts, present the benefits and communicate, communicate, communicate. Never shut up about that roadmap.”
“The biggest mistake [that new localization managers make] is not aligning with the goals and principles of the company you were hired into. Wanting to build things the right way, the localization way. You’re hired, and you’re paid, you're employed to do things for the business,” said Oleksandr.
Robert, Product Program Manager @ HubSpot, argued that you should reduce the complexity across functions by ensuring that you are unified on a single organizational value framework. “I think one of the main challenges you run into in a company is that sometimes teams can work at cross-purposes,” said Robert.
HubSpot’s framework: “What’s the value to the customer? That question starts to bring everybody into alignment. At the end of the day, if you’re not building things your customers need, neither of those matter.”
3. Ask not what your stakeholders can do for you…
By the same token, you can win the trust and appreciation of your stakeholders by looking at their metrics in closer detail.
“I think you need to identify the KPIs of your stakeholders,” argued Iggy, Localization Content Manager @ Deliveroo, “and help them to be successful eventually. That's the only way that you can start changing the attitude of your stakeholders as well – who might think about you as a contributor.”
This mirrored the advice of Francesca, Head of Internationalization @ Pinterest. “When I have [the NPS] for all the markets I’m serving, I can finally start to speak the same language that a PM is speaking.”
Oleksandr, the Localization Operations Lead @ Shopify, said: “learn the language of your internal customers.”
There is a point where generating value for stakeholders needs to be more than reactive. Imagine for a second that your function is an external agency hired by your organization and that your job within it is the account manager.
If you were trying to grow the account, you would need to sell the value of your work. That would involve not only identifying key success KPIs, but also suggesting new lines of work. Unfortunately, sales is a part of every job – yours too.
4. Don’t neglect the cost in your ROI calculations
It’s important to not simply advocate for l10n in a blanket way without referring to the cost.
That’s for two reasons. First, “you can definitely get to a level with too much complexity.” Argued Iggy, Localization Content Manager @ Deliveroo. Nearly all interviewees tiered localization investment by country; there is no way to make practical decisions without tracking costs.
But second, it’s vital to advocate for initiatives; and it reinforces the idea that localization is not a commodity but an investment with a return.
Francesca, Head of Internationalization @ Pinterest, described the way that they calculated cost for the decision to make native [non-localized] content.
“We partnered with local teams and teams across the company to understand their needs. Local teams lamented the fact that our narratives were very US-centric. But we also have limited resources.”
“So we ran a number of pilots on some specific programs – we hired copywriters, put together a workflow, so that we’re able to create native content. Now we create native content for 10 markets."
You can also calculate the opportunity cost more accurately. Iggy, Localization Manager @ Deliveroo points out: “the opportunity cost is huge. It would be super interesting to measure the difference between localization levels of a new non-English spelling market. What’s the impact of not localizing or translating any of the product?”
From the interviews: watch Iggy from Deliveroo discuss the difficulties of calculating value and the measuring impact
5. Focus on generating an international data picture
Francesca, the Head of Internationalization @ Pinterest explains the problem:
“Sometimes, you have access to very little data in localization. Dashboards are built by the US teams for the US teams. So, if you manage to have access to an international dashboard, that’s going to advocate for your initiatives.”
Using data, “I can corroborate my strategic instinct… if you ask to develop a feature, the data has to corroborate what you’re asking for.”
Here’s our advice for getting the data to proceed with recommendations. 👇
Making basic, low-fiction, tactical recommendations
The next chapter will be all about recommendations. But as we come to talk about international data, it’s useful to start with recommendations which don’t have lots of friction.
Let’s start with something easy. The first stage of your l10n recommendation engine should be to identify quick-wins and low-hanging fruit. Find the lowest-friction, highest-return recommendation you can make, find a local, short-term, high-downside problem and identify the cause.
Here’s the kind of statement we’re looking for:
“There’s low checkout completion in France”
The problem is local, because it seems to be specific to France
The problem is product-local, because it seems to be specific to the checkout
It’s short-term because the checkout is not a lag indicator. Something like churn or NPS may give a less precise indication of what could be at fault
It’s high-downside because poor checkout conversion has a clear, measurable impact on a metric somebody cares about
It’s a problem because it’s on the “objective” end of the objective-to-subjective continuum. This is a mistake, a bug, or something done poorly
The what vs the why
Many of the dashboards built by local professionals focus on the “what”, which is to build a data picture of performance in different markets. For the “why”, it depends a little on your business how to calculate it. For example, HubSpot described an engaged B2B user base which were happy to act as beta testers and would communicate their issue in some detail.
But other teams, we need a “local story” – your explanation for why a localization activity might have an effect on a metric.
But a “local story” could be something broad, or could be something narrow. Here’s a shortlist.
Narrow, functional suggestions
Local functional bugs, availability issues, integration issues, translation mistakes, [some] legal issues, currency availability, overlong & truncated strings, hardware access issues including e.g. bandwidth, WiFi.
The messy middle: LQA
Look and feel post-translation issues, insensitive media & user sentiment risks, translation flow or inconsistent translations, assumptions made during onboarding.
Broad user problems
The product and marketing doesn’t feel like it’s designed for local users, including things like US-centric case studies & marketing.
As a side effect, we can see that the role of quality has changed. In the “narrow functional bugs” space, localization is binary and commodified, and a quality issue is a mistake. In the “broad user problems” area, localization is about users, and quality is about user fit.
Tests appropriate for low-friction recommendations
Localized product flow testing
One of the most popular kinds of tests on live products that we conduct for localization teams at Global App Testing is localized user flow testing. This is common to businesses of all sizes, but investment and strategy-stage businesses tend to do product flow testing as a matter of routine.
To do this, send test cases [steps a tester should follow during a test] to real individuals in desired locations via the Global App Testing crowd. We set these projects up as a mixture of local user survey, functional test, and LQA, in order to give a 360 degree view of why performance is poor.
Where you’re unable to isolate the issue so precisely – e.g. to the checkout – we’d recommend running exploratory tests. With the same mixture of survey, functional exploratory, and LQA, you can find possible explanations for poor metrics in a 48-hour timeframe over the weekend.
Our advice to do these well
👉 Real users in real places | It can be tempting to try to run the functional element of this kind of QA out of an emulator. But functional bugs can be hard to emulate. Choose QA which involves routing your test through the country or region in question.
👉 Native speakers and cultures | To execute on the LQA element to the highest standard, you should focus on people experienced in local language & culture. We summarized our LQA approach in chapter 2, but be sure to find testers who are genuinely local as well as operating locally.
❌ Avoid internal panel building which can take a long time | For a fast or iterative product environment, panels are too slow; and they are also typically not sufficiently specialized to give practical, actionable feedback. By using a local software crowd, you can go faster, cheaper, and more focused than a local panel.
A free localization leadership webinar by Global App Testing featuring industry leaders from some of the world’s biggest brands.
Key takeaways from Chapter 3
Align to the value-add KPIs of your organization
Identifying the KPIs which make a difference to your organization and stakeholders and express your value-add in those terms
Make tactical recommendations to help people
Before you start making deep user-oriented recommendations, start with narrow and bug-oriented QA to find recommendations which are easy to enact.
Get your hands on data
It’s absolutely key to get your hands on data. Work with analysts, build local performance dashboards, and undertake investigative testing, to deliver the best recommendations.