A case for AI-enabled water management

The Wadhwani Institute for Artificial Intelligence is an independent nonprofit institute developing AI-based solutions for underserved communities in developing countries.
Accessing existing farming expertise could lay the foundation for building viable and scalable AI applications for agriculture.

Promotional adverts about technology in agriculture depict AI as sci-fi made real – Drones target diseased crops for pesticide spraying – high-definition cameras enable Combine harvesters to maximize the grain chopped from each stalk.

In reality, as we know through Wadhwani AI’s work in agriculture, AI is still in training. We are at an early stage where machine learning algorithms are used primarily for multivariate analyses.  It will take some time before the information they gather can optimize decision-making in planting, irrigation, inputs application, and harvesting. The volume and quality of training data are missing. For instance, a computer vision-based AI application that detects pests (such as our pest management solution) or plant diseases in user-generated photos needs to be trained on a large and diverse dataset of pest images. These images often contain variable lighting, angles, and background colors. Only after it is trained can it find patterns across new images that it might have never seen before and identify pests in a given farm.

Accessing existing farming expertise could lay the foundation for building viable and scalable AI applications for agriculture. By leveraging a network of experts and farmers, the AI model can learn from a broad set of relevant data and images. The output can then supplement a farmer’s knowledge and experience to help them make smarter decisions, such as the right formula and quantity of pesticide to spray at what stage of a pest infestation.

AI can thus be leveraged to deliver targeted, personalized and relevant insights and recommendations to farmers. With this goal in mind, IDH, Wadhwani AI, and Dalberg Design collaborated to identify high-potential opportunities for AI in smallholder agriculture. Water management was prioritized as an area that held considerable promise to benefit smallholder farmers. These approaches are detailed in a recent whitepaper AI Holds Water: A case for AI-enable water management in agriculture that also explores how they might be brought to the hands of farmers, programs, and policymakers, to make smarter crop decisions and drive a reduction in water usage.

Challenges & Opportunities in Water Management

Through a series of workshops, we hosted with water management experts, we tried to understand the behaviors that influence the farmer decision-making process. These can be summarized as 1) Historical patterns and precedents and community or government-recommended practices; 2) Individual, short-term benefits and profitability. For smallholder farmers, the outlook for profitability is guided by more easily quantifiable drivers (e.g., market prices or labor costs) over those that are harder to quantify (e.g., water or soil conditions). Due to limited resources and lack of appropriate planning tools, farmers prioritize maximizing their short-term, individual benefits over longer-term, collective benefits.

The next step was to identify opportunities within water management that showed potential for AI-based interventions:

1.     Water balance assessment and crop planning

For farmers, water balance assessments are useful in multiple ways, from informing crop selection to assessing the potential for multiple crops. For villages and districts, they help identify locations and plan community engagement around shared resources such as farm ponds. For states, they can help with crop planning, irrigation projects, and farmer incentive schemes, in addition to supporting other needs like fishery planning. Crop selection and planning today are strongly informed by the ease of access to markets and prices, as these are core drivers of income. However, they have an imperfect and imprecise understanding of profitability given the range of factors that influence yield volume, costs, and market prices. The influence of water availability, quality, and usage are often overlooked, as these are hard to measure precisely.

2.     Managing irrigation schedules

Limited understanding of the correlation between irrigation patterns and profitability drivers holds farmers back from making informed irrigation schedules. Like with crop selection, farmers tend to rely on historical evidence and traditional agricultural knowledge when deciding irrigation schedules. While such irrigation practices may be appropriate for some crops, in other cases, even slight changes in seed varieties can necessitate different irrigation approaches, leading to under-or over-irrigation. Irrigation also has a significant collective action problem. Farmer organizations such as collectives, coops, and SHGs make group decisions on irrigation strategies. But in practice, farmers are not incentivized to irrigate for collective benefit and tend to over-irrigate their farms using limited water resources.

How AI can make a difference

Several attempts have been made to apply AI-based solutions to water assessment. For instance, to forecast the groundwater table in agricultural land and to optimize groundwater utilization and management. Similarly, in irrigation scheduling, agtechs have leveraged sensors and remote sensing, coupled with AI, to bring smarter irrigation decisions to the market. Many of these services are neither scalable nor affordable for farmers. However, there are a few enablers that could change the game:

1. Demonstrate commercial potential

Partnership models such as PPPs, donor-driven innovation support and market strategies such as product bundling could be employed across the agtech ecosystem. These would help bring AI solutions from the research community to market and fields with the required financing. Since profitability is a key driver for farmers, affordable solutions with a clear value proposition could appeal more to them.

2. Build trust in technology

Low literacy, connectivity, access and affordability are all deterrents to adoption. To increase uptake amongst farmers, mechanisms that build trust and awareness about new technologies and their impact on farms and livelihoods are essential. Human-centered and behavioral design methodologies can be employed to orchestrate the entire farmer journey, from awareness to onboarding to sustained engagement. Farmer personas and journey mapping approaches help refine and focus products for greater reach and engagement.

3. Incorporate participatory methods

Water management in agriculture lacks data that is reliable, timely and sufficient, the lynchpins for successful AI solutions. Data acquisition through crowd-sourced and participatory mechanisms could help collect richer contextual data. With low-tech tools and training, farmers can gather groundwater data through participatory mapping (e.g., via farmer-friendly piezometre tubes) and assess soil moisture (through soil testing kits or sensors). Hyper-local data on rainfall and other weather indicators can be made available through various remote-sensing technologies.

4. Increase access to data

Ethiopia’s ATA (Agricultural Transformation Agency) and Kenya’s KALRO (Kenya Agricultural & Livestock Research Organization) are investing significantly in the development of agricultural data hubs to consolidate their own data, global datasets and local partner data to better support technology innovation across the ecosystem. Building agricultural data stacks coupled with incentives and infrastructure for sharing information (e.g., public/private clouds with appropriate computational capacity) could open up the space for innovation.

As with any new technology, getting these enablers right will take time and effort. Donors, technology innovators, agribusiness and agricultural programs, and governments all have the role to play in making this happen. We hope the white paper inspires areas where AI can be further honed to help underserved farmers optimize water management.

Curious to learn more? Check out the whitepaper here.

Co-Authors:
Ankur Seth, Program Manager, Digitalization of Agriculture, IDH
Ravi Chhatpar, Co-founder & Partner, Dalberg Design

ML Engineer

ROLES AND RESPONSIBILITIES

An ML Engineer at Wadhwani AI will be responsible for building robust machine learning solutions to problems of societal importance; usually under the guidance of senior ML scientists, and in collaboration with dedicated software engineers. To our partners, a Wadhwani AI solution is generally a decision making tool that requires some piece of data to engage. It will be your responsibility to ensure that the information provided using that piece of data is sound. This not only requires robust learned models, but pipelines over which those models can be built, tweaked, tested, and monitored. The following subsections provide details from the perspective of solution design:

Early stage of proof of concept (PoC)

  • Setup and structure code bases that support an interactive ML experimentation process, as well as quick initial deployments
  • Develop and maintain toolsets and processes for ensuring the reproducibility of results
  • Code reviews with other technical team members at various stages of the PoC
  • Develop, extend, adopt a reliable, colab-like environment for ML

Late PoC

This is early to mid-stage of AI product development

  • Develop ETL pipelines. These can also be shared and/or owned by data engineers
  • Setup and maintain feature stores, databases, and data catalogs. Ensuring data veracity and lineage of on-demand pulls
  • Develop and support model health metrics

Post PoC

Responsibilities during production deployment

  • Develop and support A/B testing. Setup continuous integration and development (CI/CD) processes and pipelines for models
  • Develop and support continuous model monitoring
  • Define and publish service-level agreements (SLAs) for model serving. Such agreements include model latency, throughput, and reliability
  • L1/L2/L3 support for model debugging
  • Develop and support model serving environments
  • Model compression and distillation

We realize this list is broad and extensive. While the ideal candidate has some exposure to each of these topics, we also envision great candidates being experts at some subset. If either of those cases happens to be you, please apply.

DESIRED QUALIFICATIONS

Master’s degree or above in a STEM field. Several years of experience getting their hands dirty applying their craft.

Programming

  • Expert level Python programmer
  • Hands-on experience with Python libraries
    • Popular neural network libraries
    • Popular data science libraries (Pandas, numpy)
  • Knowledge of systems-level programming. Under the hood knowledge of C or C++
  • Experience and knowledge of various tools that fit into the model building pipeline. There are several – you should be able to speak to the pluses and minuses of a variety of tools given some challenge within the ML development pipeline
  • Database concepts; SQL
  • Experience with cloud platforms is a plus
mle

ML Scientist

ROLES AND RESPONSIBILITIES

As an ML Scientist at Wadhwani AI, you will be responsible for building robust machine learning solutions to problems of societal importance, usually under the guidance of senior ML scientists. You will participate in translating a problem in the social sector to a well-defined AI problem, in the development and execution of algorithms and solutions to the problem, in the successful and scaled deployment of the AI solution, and in defining appropriate metrics to evaluate the effectiveness of the deployed solution.

In order to apply machine learning for social good, you will need to understand user challenges and their context, curate and transform data, train and validate models, run simulations, and broadly derive insights from data. In doing so, you will work in cross-functional teams spanning ML modeling, engineering, product, and domain experts. You will also interface with social sector organizations as appropriate.  

REQUIREMENTS

Associate ML scientists will have a strong academic background in a quantitative field (see below) at the Bachelor’s or Master’s level, with project experience in applied machine learning. They will possess demonstrable skills in coding, data mining and analysis, and building and implementing ML or statistical models. Where needed, they will have to learn and adapt to the requirements imposed by real-life, scaled deployments. 

Candidates should have excellent communication skills and a willingness to adapt to the challenges of doing applied work for social good. 

DESIRED QUALIFICATIONS

  • B.Tech./B.E./B.S./M.Tech./M.E./M.S./M.Sc. or equivalent in Computer Science, Electrical Engineering, Statistics, Applied Mathematics, Physics, Economics, or a relevant quantitative field. Work experience beyond the terminal degree will determine the appropriate seniority level.
  • Solid software engineering skills across one or multiple languages including Python, C++, Java.
  • Interest in applying software engineering practices to ML projects.
  • Track record of project work in applied machine learning. Experience in applying AI models to concrete real-world problems is a plus.
  • Strong verbal and written communication skills in English.
mls