We ran a pilot remotely, in the midst of a pandemic. It worked

Getting anything done in the pandemic is tricky. We managed to get our early pest management system off the ground during Covid19 lockdowns.
Farmers despite not being friendly with technology found the app easy to use.

Running through treacle. It’s the best way to describe March. Everything was an effort. Getting the most basic message across took time. For the products and programs team, it felt like any effort was wasted. And they almost gave up. 

In March, the Wadhwani Institute for Artificial Intelligence was set to run its pilot program on cotton pest management. The solution looks pretty simple from the outside. Farmers set up traps, monitor, install an app, take a picture, AI gives them a suggestion, they implement the suggestion, they manage to ward off pest attacks. It’s almost like reciting the alphabet. 

But in reality, the alphabet is difficult, especially when you both speak different languages. The English alphabet has 26 letters. Kannada, a language spoken in the South of India, has around 50. Not just language, how do you teach a 35-year-old farmer how to use an app when all they don’t even have a smartphone. Even if somehow you could get through, find a way to process their images, how do you determine that the farmers actually follow your advice? 

These are challenges even when you aren’t asked to shelter in place. The solution before the pandemic? The team would split into small groups and go to different villages in districts, onboard farmers, set up processes and then return. But how do you handle it in a lockdown? Even with masks, the Coronavirus can infect people. Apart from the health risk is the logistical issue, how does one get around during a lockdown? This part of the job couldn’t be done with social distancing. 

But when things get weird, the weird turn pro.

There is always an answer

There is a simple solution. Hire. But hiring comes with challenges. How do you describe a program and its nuances at its pilot stage to an entirely new set of people? Even though the basics are easy, the key here is not just explaining the solution to a farmer but also to convince her that the phone is giving her the right answer. Farming in India, to a large extent, is an occupation that is inherited and so are the methods. It is difficult to convince farmers that the times have changed and their thinking needs to evolve. New problems need new solutions. And these solutions are hidden in technology. 

“We have to be careful not to overcommit,” says Rajesh Jain, senior director, programs, Wadhwani Institute for Artificial Intelligence. The solution can’t be the pest equivalent to the one ring

The team had to reimagine the entire pilot. So they started small. Let’s hire a manager. But a manager that is in the district. This is a tricky task. Finding experts who not only can speak the language but are geographically close and understand pests and cotton is difficult. Fortunately, in a team’s earlier visit in February, they had managed to make connections with officers in the agriculture department. This proved to be a valuable link. The officers helped the team find the right manager. Now the plan was to get this manager to go to individual villages, talk to sarpanch (elected leader of a village), get consent and find out how many farmers could be interested. While the sarpanch gathers the farmers and gauges interest levels, the manager starts to hire a team. 

The pitch by the manager: The job is on a short term contract and is going to require a lot of travel during a pandemic. But you help cotton farmers and teach them to install the traps and use the app properly on their phone. 

And it worked. We managed to hire three extension workers. The sarpanch and lead farmers of different villages helped with the numbers and contacts of farmers. Almost 120 farmers volunteered to try the solution. The extension workers were assigned to the farmers. The process was now split into two parts. One would be at the village level.

This is where the extension workers and the manager would be the face of the institute. They would be responsible for whipping up excitement. The team would work quietly in the background course-correcting when needed. For the sake of the pilot, the team split the farmers into four groups.

How often do the farmers meet with the staff in the pandemic?

Now it gets tricky. Everything the team made was in Mumbai. Hubli is a small district in Karnataka about 570km from the Institute’s headquarters. The languages spoken in both states are different. Even the script is different. To add to this, everything made by the team was in English. 

“Google translate was our answer. English was converted to Kannada, our onboarding posters and charts were translated. Our advice on the phone was translated too,” says Rajesh.

It is a neat workaround until you realise that language throws up funny quirks. “Our in-app advice to a farmer involved us giving him an option to choose between three types of pesticide,” says Rajesh. Something broke between the translation and interpretation and the farmer interpreted the or as and tried to find all three pesticides. “He couldn’t find all three so he called us and told us that our advice didn’t work,” adds Rajesh. 

This was important learning. It meant there was more training needed. 

The team’s idea was to create a video. 

A video would be self-explanatory. But the language was a problem once again. “The managers came up with their own solution here,” says Rajesh. The manager watched the video, understood the process and went to the farms. 

Here he repeated the processes in the video but he spoke in Kannada while explaining the steps to the farmers. This entire process was recorded and posted to a WhatsApp group. The video was watched by most farmers in the group.

Two-factor authentication

The WhatsApp group plays an important role in the pilot. “Not only was it a one-point contact for our pilot, but we could also educate the farmers about masks and social distancing,” says Rajesh. In the first few months of the virus hitting India, smaller districts were aware of the virus but not of the seriousness primarily because the infection hadn’t left the cities. This WhatsApp group convinces farmers that if they follow the advice, it may help their crop. It is also a place for exchanging ideas, learning new techniques and a place for social interaction. 

Now with the technology in the hands of the farmers, it is important to determine if the farmers actually use it correctly and regularly. Just asking the farmers isn’t enough. So the team put in place a two-factor authentication system. The extension workers hired by the institute entered an informal partnership with the local pesticide seller. The agreement? You tell us when one of our farmers buys a particular product. Now, each time one of the farmers that were part of the pilot bought a pesticide, the institute would know. It created a feedback loop. But there is still a gap between purchase and use. 

A farmer spraying pesticide as recommended by the Wadhwani AI app

“The extension worker would take a picture or video of the farmer spraying the pesticide,” says Rajesh. This would complete the cycle. Now, the institute would know that not only was the information reaching the farmer, he was also implementing the advice. 

Setting up this entire pilot from scratch took 27 days. The team is quietly very happy with their effort. And there have been plenty of learnings too. 

“If I could go back in time, I would fix our communication. Every time we slowed down it was because we couldn’t get our message across. It was usually language issues. You can only go so far with videos and Google,” says Rajesh. 

So what would you do differently? Hire someone who understood a little more English. It is a minor quibble though, Rajesh admits. It is a complex set of instructions have to be delivered remotely, even if the manager was fluent in English.

Rajesh believes that this is not just a successful pilot for Wadhwani Institute for Artificial Intelligence but also for anyone who wants to run an agri-tech program in the developing world. “Look, the virus isn’t going anywhere soon. We will have to learn to conduct our programs remotely. And this is going to be the blueprint going forward,” says Rajesh. 

His advice to anyone attempting something similar? “The time is ripe to use technology to solve old problems, you just need to have the conviction,” Rajesh adds. 

  • Wadhwani AI

    We are an independent and nonprofit institute developing multiple AI-based solutions in healthcare and agriculture, to bring about sustainable social impact at scale through the use of artificial intelligence.

ML Engineer

ROLES AND RESPONSIBILITIES

An ML Engineer at Wadhwani AI will be responsible for building robust machine learning solutions to problems of societal importance; usually under the guidance of senior ML scientists, and in collaboration with dedicated software engineers. To our partners, a Wadhwani AI solution is generally a decision making tool that requires some piece of data to engage. It will be your responsibility to ensure that the information provided using that piece of data is sound. This not only requires robust learned models, but pipelines over which those models can be built, tweaked, tested, and monitored. The following subsections provide details from the perspective of solution design:

Early stage of proof of concept (PoC)

  • Setup and structure code bases that support an interactive ML experimentation process, as well as quick initial deployments
  • Develop and maintain toolsets and processes for ensuring the reproducibility of results
  • Code reviews with other technical team members at various stages of the PoC
  • Develop, extend, adopt a reliable, colab-like environment for ML

Late PoC

This is early to mid-stage of AI product development

  • Develop ETL pipelines. These can also be shared and/or owned by data engineers
  • Setup and maintain feature stores, databases, and data catalogs. Ensuring data veracity and lineage of on-demand pulls
  • Develop and support model health metrics

Post PoC

Responsibilities during production deployment

  • Develop and support A/B testing. Setup continuous integration and development (CI/CD) processes and pipelines for models
  • Develop and support continuous model monitoring
  • Define and publish service-level agreements (SLAs) for model serving. Such agreements include model latency, throughput, and reliability
  • L1/L2/L3 support for model debugging
  • Develop and support model serving environments
  • Model compression and distillation

We realize this list is broad and extensive. While the ideal candidate has some exposure to each of these topics, we also envision great candidates being experts at some subset. If either of those cases happens to be you, please apply.

DESIRED QUALIFICATIONS

Master’s degree or above in a STEM field. Several years of experience getting their hands dirty applying their craft.

Programming

  • Expert level Python programmer
  • Hands-on experience with Python libraries
    • Popular neural network libraries
    • Popular data science libraries (Pandas, numpy)
  • Knowledge of systems-level programming. Under the hood knowledge of C or C++
  • Experience and knowledge of various tools that fit into the model building pipeline. There are several – you should be able to speak to the pluses and minuses of a variety of tools given some challenge within the ML development pipeline
  • Database concepts; SQL
  • Experience with cloud platforms is a plus
mle

ML Scientist

ROLES AND RESPONSIBILITIES

As an ML Scientist at Wadhwani AI, you will be responsible for building robust machine learning solutions to problems of societal importance, usually under the guidance of senior ML scientists. You will participate in translating a problem in the social sector to a well-defined AI problem, in the development and execution of algorithms and solutions to the problem, in the successful and scaled deployment of the AI solution, and in defining appropriate metrics to evaluate the effectiveness of the deployed solution.

In order to apply machine learning for social good, you will need to understand user challenges and their context, curate and transform data, train and validate models, run simulations, and broadly derive insights from data. In doing so, you will work in cross-functional teams spanning ML modeling, engineering, product, and domain experts. You will also interface with social sector organizations as appropriate.  

REQUIREMENTS

Associate ML scientists will have a strong academic background in a quantitative field (see below) at the Bachelor’s or Master’s level, with project experience in applied machine learning. They will possess demonstrable skills in coding, data mining and analysis, and building and implementing ML or statistical models. Where needed, they will have to learn and adapt to the requirements imposed by real-life, scaled deployments. 

Candidates should have excellent communication skills and a willingness to adapt to the challenges of doing applied work for social good. 

DESIRED QUALIFICATIONS

  • B.Tech./B.E./B.S./M.Tech./M.E./M.S./M.Sc. or equivalent in Computer Science, Electrical Engineering, Statistics, Applied Mathematics, Physics, Economics, or a relevant quantitative field. Work experience beyond the terminal degree will determine the appropriate seniority level.
  • Solid software engineering skills across one or multiple languages including Python, C++, Java.
  • Interest in applying software engineering practices to ML projects.
  • Track record of project work in applied machine learning. Experience in applying AI models to concrete real-world problems is a plus.
  • Strong verbal and written communication skills in English.
mls