- Digitizing public services does not guarantee better results: without evidence, many solutions fail to improve the citizen experience.
- Tools that reduce friction (such as simple reminders or easy scheduling) have a greater impact than complex technological solutions or persuasive messaging.
- Experiments allow us to identify what works before scaling up, helping to design more effective, people-centered digital policies.
Digitizing public services now seems to be the automatic response to any problem: if a service has low adoption rates, digitize it. If administrative costs are high, digitize it. If communication with citizens is poor, digitize it. But digitization is not the same as transformation, and it does not always address people’s real needs.
In recent years, governments in Latin America and the Caribbean (LAC) have invested significantly in platforms, mobile apps, and digital identity. However, we still know little about which digital tools actually improve people’s lives. Unlike sectors such as health or education, where rigorous evaluations abound, the field of digital government is still building its evidence base.
To this end, the key question is not whether we should digitize, but how to better measure the impact of digital government before scaling up solutions. Helping governments answer this question is part of our work agenda at the Inter-American Development Bank (IDB).
The evaluation of digital government tools faces three main challenges:
1. Multiple changes occur simultaneously. When a new digital platform is implemented, it is often part of a broader package of reforms, making it difficult to isolate the impact of that specific intervention. If the number of appointments increases after launching an online medical appointment system, was it due to the technology or the additional staff hired at the same time?
2. Digital users do not represent the entire population: Those who use digital services tend to have better internet access, stronger digital skills, and higher levels of education. This means that the results of digitization policy implementation cannot necessarily be generalized to the entire population.
3. More data does not always mean better results: Digitization improves data collection, which can make it appear as though there is more activity when in reality there is only better measurement.
One of the most reliable tools for isolating the actual effect of a digital intervention is the randomized controlled trial (RCT). In an RCT, citizens or government officials are randomly assigned to different groups: some receive the new digital tool, while others do not. This comparison makes it possible to identify more precisely what works and what does not.
The following four cases in the region illustrate how this methodology can improve decision-making in digital government.
1. Panama: Text Message Reminders Increase ID Card Renewals
The challenge: 15% of people with expired ID cards did not renew them on time, affecting their access to public services.
The experiment: To increase the number of people renewing their ID cards on time, text messages were sent to more than 3,000 people, randomly assigned to three approaches:
-Text message (SMS) reminder
-SMS reminder + web link with pre-filled information to schedule an appointment
-Control group (no intervention)
The result: The simple SMS reminder was more effective than the digital link. Nearly 60% of those who attempted to use the platform abandoned the process. The main obstacle was the requirement to take a selfie that met specific technical criteria.
Key lesson: More technology does not always mean better results. When a tool adds friction, it can reduce adoption. Sometimes, simple is better.
The challenge: 60% of women in Montevideo, Uruguay, were not adhering to cervical cancer screening recommendations.
The experiment: 47,600 women were randomly assigned to one of four groups:
- Message emphasizing the benefits of screening, accompanied by access to digital scheduling
- Message emphasizing the risks of not getting screened, accompanied by access to digital scheduling.
- A message emphasizing the benefits of screening, without access to digital scheduling
- A message emphasizing the risks of not getting screened, without access to digital scheduling
The result: Persuasive messages (nudges) had no significant impact, but offering a digital tool to facilitate scheduling tripled appointment attendance (from 1.9% to 5.3%).
Key takeaway: Technology that reduces friction has a greater impact than persuasive messages.
3. Uruguay: Personalized Promotion Boosts Awareness, but Not Adoption of Digital Identity
The challenge: While digital identity offers many benefits, adoption remains limited.
The experiment: With a sample of 15,000 people, three channels were tested to promote digital ID adoption: SMS (one-way), WhatsApp (two-way and personalized), and phone calls (two-way and personalized).
The result: WhatsApp was the most cost-effective channel for raising awareness, with 17 percentage points more people recalling having received digital ID promotion. But no channel managed to increase adoption. There is also significant confusion regarding ownership: 20% of those without a digital identity believed they had one, while 35% of those who did have one believed they did not.
Key takeaway: Raising awareness does not guarantee adoption, especially when the process is complex.
4. Argentina: Hands-On Simulations Improve Cybersecurity in the Public Sector
The challenge: More than two-thirds of cyberattacks result from human error. Phishing (malicious emails) is the most common type, and research suggests that 21% of employees could fall victim to an attack.
The experiment: Two training approaches were evaluated with 1,920 civil servants:
-Traditional 40-minute courses
-Practical phishing simulations with immediate feedback
The result: The practical approach reduced susceptibility to phishing by 7 percentage points, while the traditional approach achieved only a 2-point reduction.
Key takeaway: In public cybersecurity, hands-on experience trumps theoretical training.
Experiments are particularly useful when:
- There are multiple options for promotion or implementation, but it is unclear which one is best.
- You plan to conduct a pilot before scaling up a digital tool.
- The implementation of the tool is planned to be gradual.
- The goal is to measure the actual impact and magnitude of the effects of digital tools on the citizen experience.
On the other hand, they are not the best tool when:
- A decision must be made regarding a massive investment. The goal is to evaluate a program that will be available nationwide.
- Effects are only expected to be seen in the long term.
- There is no institutional appetite for using control groups, generally due to the risk of excluding some potential beneficiaries.
In summary, experiments are most valuable when they help refine and learn, but not when the goal is to resolve the entire public policy issue at once.
The digital transformation of government will continue to accelerate in Latin America and the Caribbean. However, its effectiveness is not guaranteed.
The evidence gathered from these experiments suggests three key principles:
1. Reducing friction matters more than persuasion.
2. Simplicity often trumps technological sophistication.
3. Hands-on experience can change behaviors more than theoretical information.
Before scaling up a new digital tool, governments should ask themselves:
Do we have evidence that this actually improves people’s lives?
Piloting, measuring, and adjusting do not slow down public innovation; on the contrary, they strengthen it. Each RCT enriches global knowledge about what works to improve the citizen experience and public service.
That is why, at the IDB, we promote an evidence-based digital government agenda that combines rigorous research, controlled pilots, and impact evaluations.
Through these experiments, we help governments make more informed decisions before scaling up technological solutions and facilitate the transfer of lessons learned among countries in the region. Generating evidence on what reduces friction, what changes behavior, and what does not produce the expected results allows for the design of more strategic and citizen-centered digital policies.
Ultimately, it is not about digitizing for the sake of digitizing. It is about building a digital government that focuses on improving the lives of its citizens.
Keywords:
Public Administration