Building a Crop Yield Prediction App Using Satellite Imagery and Jupyter

Omdena´s Crop Yield Prediction AI Challenge in Africa

In this Omdena AI Challenge with the Global Partnership for Sustainable Development Data, we created a simple but powerful application using GEE images to estimate crop yield in Senegal.

For food security understanding the food system is essential. Accurate crop type details in near real-time will provide policymakers insights on the food system and will provide information on crop diversity and nutrition outcomes. So we created an application using open-source satellite images to identify the crops and estimate the yields for any given area.

Challenges Encountered

1. The first challenge is to acquire satellite images from the regions selected by the user interactively over a map.

For this, we came up with the Python package Geemap which allows users to select a region in the map and get the geo-coordinates of the selected region.

!pip install geemap

import geemap
Map = geemap.Map()

#draw a rectangle in the above map and then run this
#ROI - Region of Interest

feature = Map.draw_last_feature
ROI = feature.geometry()
ROI.getInfo()

Output:

{'geodesic': False,
 'type': 'Polygon',
 'coordinates': [[[-120.021748, 46.126847],
   [-120.021748, 46.126957],
   [-120.021535, 46.126957],
   [-120.021535, 46.126847],
   [-120.021748, 46.126847]]]}

2. Downloading the geo tiff file and processing the file to get the data in the form of a matrix is computationally powerful.

We used the Geemap function ‘ee_to_numpy’ for this. It converts the image collected from the Google earth engine ‘ee.image’ into ‘NumPy’ array. It doesn’t need local computational power. It is all done in Google Earth Engine itself.

import ee

def maskS2clouds(image):
  qa = image.select('QA60')
  cloudBitMask = 1 << 10
  cirrusBitMask = 1 << 11
  mask = qa.bitwiseAnd(cloudBitMask).eq(0).And(
             qa.bitwiseAnd(cirrusBitMask).eq(0))
  return image.updateMask(mask).divide(10000) 


# Load Sentinel-2 TOA reflectance data.
imageCollection = ee.ImageCollection('COPERNICUS/S2') 
    .filterDate('2016-01-01', '2016-12-31') 
    .filter(ee.Filter.lt('CLOUDY_PIXEL_PERCENTAGE', 5)) 
    .map(maskS2clouds) 
    .filterBounds(roi) 
    .sort('CLOUDY_PIXEL_PERCENTAGE')


rawImg = ee.Image(imageCollection.toList(imageCollection.size()).get(0))

#get the matrix for B2 band only
B2_image = geemap.ee_to_numpy(rawImg.select(['B2']), region=ROI)

3. Each band in the sentinel is of a different resolution. So the matrix we got is of different shapes for each resolution. We wanted to have all the matrices stacked on top of each other to get a single multi-channel image.

We resized all the matrix to the high resolution (In this case 10 meters resolution) using the ‘nearest neighbor interpolation’ method.

targetSize = (50,50)

B5_Matrix = cv2.resize(B5_image,targetSize,interpolation=cv2.INTER_NEAREST)

Creating the Crop Yield Prediction Application

To create the crop yield prediction app we used ‘ipywidgets’ which is an interactive HTML widget for Jupyter Notebook, Jupyter Lab, and Ipython Kernel. Then we used Voilà to turn the Jupyter notebooks into standalone web applications.

Step 1:  Define the user interface

Sample code

#Date picker 
dateBox = widgets.DatePicker(
    description='',
    disabled=False
)

#Map
mapWidget = widgets.Output()
    
#labels
step1 = widgets.Label('Step 1: Select the date')
step2 = widgets.Label('Step 2: Select Region from the map')
step3 = widgets.Label('Step 3: Load model')
step4 = widgets.Label('Step 4: Estimate yield')
estimate_yield_debug = widgets.Label('')

#Buttons
getROI = widgets.Button(description='Ok')
estimate_yield_Btn = widgets.Button(description='Estimate yield')
loadModel_Btn = widgets.Button(description='Load model')

#Progress bar
progressBar = widgets.IntProgress(
    value=0,
    min=0,
    max=19,
    step=1,
    description='',
    bar_style='info', 
    orientation='horizontal'
)

#Text Area
estimate_yield_Out = widgets.Textarea(
    value='',
    placeholder='',
    description='',
    disabled=True,
    layout={'height': '300px'}
)

#Display prediction image (Matplotlib plot)
estimate_yield_plot = widgets.Output()

Step 2:  Updating the widgets

1. Updating a label

for band in bands:
    estimate_yield_debug.value = f"Processing: {band}"

2. Reading information from date Picker

year  = dateBox.value.year
month = dateBox.value.month
day   = dateBox.value.day

3. To display Map

Map = geemap.Map(center=[14.607989,-14.531731], zoom=7)
Map.add_basemap('Google Satellite Hybrid')

with mapWidget:
    display(Map)

4. To trigger a function when a button is clicked

def getStatistics(change):
      """
      some codes

      """    
getROI.on_click(getStatistics)

5. To update the progress bar

for band in bands:
      estimate_yield_debug.value = f"Processing: {band}"

      #Update progress bar
      progressBar.value = progressBar.value +1

6. To display matplot image

def plotResult(prediction):
    """
    Code to visualize the prediction in the form of Matplotlib image.
    """

    
    #display output
    plt.show()

#Visualize the prediction
with estimate_yield_plot:
      plotResult(prediction)

Step 3:  Create the app layout

Vertical Box (widgets.VBox) and Horizontal Box (widgets.HBox) are used to group widgets together.

Then we can use AppLayout from ipywidgets to align all the widgets in a proper way.

from ipywidgets import AppLayout

#Arrange the layout
verticalBox = mapWidget
vBox1 = widgets.VBox([step1, dateBox,vBox])
vBox2 = widgets.VBox([step2, getROI,
                      step3,loadModel_Btn,                            step4,estimate_yield_Btn,progressBar,estimate_yield_debug,yieldOutput,
                      estimate_yield_Out,estimate_yield_plot])

AppLayout(header=None,
          left_sidebar=vBox1,
          right_sidebar=vBox2,
          footer=None,
          height="70%", width="150%")

Steps to run the application

1. Open the notebook

2. Click on the Voilà button

Check out the 2 mins app demo