Author of an article and illustrations: Weronika Dorocka

The Problem 

For more than a century, the energy sector has been driven by centralized grid-based systems. But 135 years after the advent of these systems, millions of people are still left behind with insufficient or no electricity. One just needs to visit a developing country to realize how transient and unreliable energy access can be.

Any move towards putting power in the hands of people has always featured the extraordinary involvement of community (think universal franchise or independence movements!).

Solar needs to become a people’s movement to realize the vision of a decentralized energy independent future.

Today, we are blessed with access to technology that gets better at solving problems with each participation by the community.

This article is the story of how we are building this future. Join us on this journey and learn how you too can contribute to an energy secure future.

The Goal

The big picture aim of a partner was to to build an app that would play an important role to make solar power more available and the knowledge behind the costs and savings that come with it more accessible. In other words to build a simulator that can help people get an estimate of their rooftop solar potential and ‘test drive’ a virtual solar plant. They wanted people to be able to place solar panels virtually on their roof and see their day to day savings. Long term vision is about simply empowering especially developing countries with access to more affordable energy. 

Our goal in this specific project was to build a model that would be able to detect precisely the rooftops area available for montaging of solar panels for domestic households.

The Background

Bright Opportunity for Energy Independence

In the case of most residences, the solar energy generated from a rooftop solar plant is sufficient to power the entire consumption of the household. There is no reason why each building cannot be self-sustaining and independent when it comes to energy.

Such systems are already providing cheap electricity to millions of homes across the world. With the growing demand for solar, we are also heading for an inflection point where solar + storage will be competitive with electricity from the existing system, and allow individual buildings to be self-sufficient for energy. With storage removing the dependence on the grid, households and communities can become self-sufficient, paving the way for an ecosystem of decentralized and clean energy.

So what stops us from immediate mass scale adoption of solar rooftops? 

A quick look at the challenges faced by a user on their solar journey will give us an idea:

1. Lack of organized relevant information: To understand what solar means for you, today’s solar journey necessarily requires a physical visit by a solar company engineer to assess your property. The few good ‘online calculators’ give a very rough approximate at best, or require a lot of data input that you may not have an idea about (eg: what is the shadow-free area on your roof?)
2. Time-consuming offline engagement process: Once an accurate assessment is made, customer education happens over a number of physical meetings with several solar companies in order to form an informed opinion. It is also difficult to leverage the experience of existing solar users, in the absence of an online solar community
3. Insufficient support: Existing solar users are the best ambassadors of solar adoption. However, as maintenance is not a lucrative option for solar companies, many solar users are left hanging and do not have a positive experience to share with potential solar adopters.

The heavy offline component involved in today’s solar process makes solar sales expensive and time-consuming. This limits the ability of solar companies to approach more people and adversely affects the growth of solar adoption. As a result, solar adoption remains a project of governments and solar companies and is not backed by a popular participation and engagement of people.

The Challenges

Identifying Objects from Low Quality Images

The critical challenge here is the ability to remotely map the features of a roof, including boundaries and obstacles so that the area suitable for solar can be identified. Project Sunroof of Google is trying to solve this problem too and has provided partial solutions for certain cities in the US and Europe. 

However, due to the bad quality of satellite imagery in India (and other developing countries), their solution is not suitable.

Identifying Critical Roof Features from a Low-Quality Satellite Image

Existing algorithms don’t really work for rooftop analysis due to the low quality of satellite images available for Indian roofs. A machine learning approach based on training roof datasets is promising to solve a hitherto unsolvable problem. In case you are not interested in a technical discussion of the same, you can choose to skip to the next section to see how you too can contribute to solving this problem.

Our Approach

Step 1. Identifying the best technological method

Taking different approaches to  be the best to detect the rooftops and their specific unique for each building metrage, knowing there are also there other elements that need to be ignored such as chimneys, water boilers etc.

The aspects that needed to be taken into account:

• Obstacles in rooftop — We have identified that obstacles on roofs typically belong to around 10 different families of obstacle types, for instance: Chimneys, Water tanks, and Turbo ventilators
• Type of roof — Roofs can also be classified into broadly a few categories, again validated by our manual mapping.
• Edges of the roof — This is the most challenging part of the problem, because we wanted to train a machine to learn to identify the edges, and to be able to distinguish one property from the other and one roof level from the other.

1. Trying the Algorithms for Differentiating Rooftops (through Image Segmentation)

We started with the OpenCV library* (Open Source Computer Vision Library) to see how the roof edges and features can be detected algorithmically. Shown below is a solar rooftop image from Germany.

* Open CV  (Open Source Computer Vision Library) is a powerful open-source library for computer vision and image processing tasks. It provides a wide range of tools and algorithms for tasks like object detection, image segmentation, facial recognition, and more.

When we tried two different building’s edge detection methods on satellite images:

1.1. Watershed Algorithm: The watershed algorithm is like pouring water on a picture to find where the colors change a lot, separating different objects or areas in the image. It helps computers understand where one thing ends and another begins.

The Watershed algorithm is especially useful when extracting touching or overlapping objects in the images. 

Below are the results from the Watershed algorithm.

1.2. Canny Edge Detection: It’s an image processing technique used to find the edges in pictures, helping computers recognize objects by detecting areas where there’s a sudden change in colour or intensity.

Example of Canny Edge Algorithm Effects:

With the quality of the satellite image, the algorithmic approach worked fairly well, and with some noise reduction, we were able to identify the edges correctly. But when we applied the same algorithm on the low quality images rooftops, the results were far worse. The image quality from residential areas was particularly bad, and feature identification was made even more difficult by the non-uniform roofs and proximity of neighbouring buildings. We aimed to analyze satellite images and differentiate rooftops from other structures, a process known as image segmentation.

Although they showed promise, their accuracy didn’t meet our standards.

2. Second and Better Approach: Making the Machines understand the Meaning behind the Picture

Both of the above techniques use Image Segmentation, but without understanding the context and content of the object we are trying to detect (i.e. rooftops)! Instead Semantic segmentation attempts to partition the image into semantically meaningful parts and to classify each part into one of the predetermined classes.

Semantic Segmentation Example:

Therefore we decided to implement another AI technique that would enforce Semantic Segmentation logic: Convolutional Neural Networks*

CNN: A Convolutional Neural Network model, or CNN in other words, is a type of artificial intelligence model inspired by the human visual system. It’s particularly good at analysing visual data like images and videos. Think of it like a smart detective that can recognize patterns and features in pictures, helping computers understand and interpret visual information.

Step 2. Breaking the Barrier of no Suitable Dataset

Our team meticulously tagged images of Indian buildings and created masks to highlight rooftops. With limited initial data, we employed creative techniques like data augmentation, which involved generating additional images through simple alterations.

In our case, each pixel of the image needed to be labelled as a part of the rooftop or not.

Another challenge was that we had only 20 images in our training set which is way below for any model to give results.

How was it possible to generate more Data out of a small amount of Images? Data Augmentation

One of the most popular techniques to deal with less data is Data Augmentation. Through Data Augmentation we can generate more data images using the ones in our dataset by adding a few basic alterations in the original ones.

For example, in our case, any Rooftop Image when rotated by a few degrees or flipped either horizontally or vertically could act as a new rooftop image, given the rotation or flipping is in an exact manner, for both the roof images and their masks. We used the Keras Image Generator on already tagged images to create more images.

Step 3. Attempt to sharpen the low quality images.

We used two different sharpening filters — low/soft sharpening and high/strong sharpening. After sharpening we applied a Bilateral filter for noise reduction produced by sharpening. Below are some lines of Python code for sharpening

Outcomes:

Step 4. Creating the final functionalities of the model

We generated training data of 445 images. Next, we chose to use U-Net architecture. U-net was initially used for Biomedical image segmentation, but because of the good results it was able to achieve, U-net is being applied in a variety of other tasks. is one of the best network architecture for image segmentation.

At some point we found out that there were some strange lines in the middle and corners of the shapes. We learned that this was because of a certain type of mistake we were making. So, we tried a different method called Adam, and it worked better. We also changed how we measured our success and found that our model was right about 86% of the time during training and about 79% of the time when we checked its guesses with new pictures.

Other Applications of such Technology

Applied Methodology 1:

Rooftop Detection through Satellite Images

Can also be utilized in:

• Real Estate company to detect where are great locations for property purchase.
• Chain of Stores can build a similar model to detect and propose best locations for the new store to be opened.

Applied Methodology 2:

Improving the Resolution of Images

Can also be utilized in:

• E-commerce: Enhancing product images for online stores to boost sales and customer satisfaction.
• Healthcare: Improving the clarity of medical images for more accurate diagnoses and treatment planning.
• Satellite Imaging: Enhancing satellite images for better mapping, urban planning, and environmental monitoring.
• Surveillance and Security: Sharpening surveillance footage for better identification of suspects and enhanced security measures.
• Media and Entertainment: Enhancing visual content in movies, TV shows, and digital artwork for better viewer experiences.
• Automotive Industry: Improving image quality for vehicle cameras to enhance safety features and autonomous driving capabilities.
• Agriculture: Enhancing aerial and drone imagery for better crop monitoring and precision agriculture.
• Archaeology and Cultural Heritage: Improving the resolution of historical photographs and archaeological imagery for documentation and preservation purposes.

Time Frame

The whole Project took just 6 months knowing we needed to:

1. Test different possible approaches and choose the best one (it was thanks to our Unique System of Innovation Challenge where 50 engineers collaborate to solve an innovative problem)
2. We needed to find, create and manually work on all the Data.

Possible Next Steps

1. Refining the Model Continuously: Implementing a feedback loop mechanism to continuously refine the AI model based on user feedback and real-world data.
2. Incorporating Weather Data: Integrating real-time weather data into the AI model to provide more accurate solar energy yield predictions.
3. Personalizing Recommendations: Developing algorithms that generate personalized recommendations for optimal solar panel placement and system configurations based on individual preferences.
4. Enhancing Visualization Tools: Enhancing the visualization capabilities of the simulator by integrating immersive technologies such as virtual reality (VR) and augmented reality (AR).
5. Implementing Predictive Maintenance: Implementing predictive maintenance solutions powered by AI to monitor the performance of solar panels and detect potential issues before they escalate.
6. Developing Data Analytics Dashboard: Developing a comprehensive data analytics dashboard that provides users with insights into their solar energy production and consumption patterns.
7. Integrating Blockchain for Energy Trading: Exploring the integration of blockchain technology to enable peer-to-peer energy trading among rooftop solar owners within a community or microgrid.
8. Utilizing Machine Learning for Demand Forecasting: Developing machine learning models for demand forecasting to predict future energy consumption patterns and optimize solar energy production and storage.
9. Automating Permitting and Regulatory Compliance: Implementing AI-driven solutions to streamline the permitting and regulatory compliance process for rooftop solar installations.

Want to work with us too?

Let’s see if we are a good fit