Author: Anber Arif. Co-author: Mahzad Khoshlessan. Peer-reviewed: Irune Lansorena
For decades, scientists and engineers have been dreaming of creating machines that can think and act like humans. They have been trying to give computers the ability to see and interpret the world around them. Thanks to the computer vision engineers as they made the fiction of yesterday the fact of today.
Advancements in artificial intelligence in recent years surpassed humans in some tasks related to the detection and labeling of objects, thanks to the amount of data we create daily as individuals. One of the main enabling factors is the mind-blowing amount of data that we generate as individuals: 2.5 quintillion bytes of data created each day at our current pace, but that pace is only accelerating with the growth of the Internet of Things (IoT). [1]
What is Computer Vision?
Computer Vision is a field of Artificial Intelligence (AI) that replicates parts of the complexity of the human vision system and creates digital systems that are able to process, analyze, and make sense of images or videos (visual data) in the same way that humans do.
Research in computer vision can be divided into three sections:
Computer Vision: Research in computer vision focuses on the underlying algorithmic, geometric, and optic issues. It also studies various image processing topics such as image formation, depth and shape recovery, photometry, motion, classification, and special topics.
- 3D Computer Vision: 3D computer vision aims to extract 3D information from image and video data. It consists of concepts relating to 2D images and 3D scenes including single and multiple-view geometry, structure from motion, and 3D reconstruction.
- Robot Perception: Research on computer vision models that enables robots to perceive, comprehend, and reason about the surrounding environment.
We can use computer vision systems for a few common tasks such as:
- Object Identification
- Object Tracking
- Object Classification
- Facial Recognition
- Image Classification etc.
Image Source: Google
Computer Vision has a wide range of applications in various sectors including health, energy, agriculture, autonomous driving, etc. Computer vision startups like Wobot.AI, Mojo, and ubble utilize the power of Artificial Intelligence to solve multiple real-world problems.
What Does a Computer Vision Engineer Do?
Computer Vision is one of the fastest growing fields that trains computers to interpret and understand the visual world. It seeks to automate the human visual system and is at the center of technological advancements.
Computer vision engineers are responsible for developing complex machine algorithms that can comprehend, analyze, and extract key information from large visual data. They use a data-based approach to develop solutions. They have experience working on a variety of systems and come with a background in Machine Learning or Artificial Intelligence.
Image Source: Google
Following are the primary responsibilities of a computer vision engineer.
- They create, annotate and handle large datasets.
- They conduct image enhancement.
- They develop and automate computer vision models.
- They interact with the engineering team and the client for innovating new products and features and incorporating real-time feedback.
- They develop and test computer vision solutions for real-life problems and applications and modify existing models.
- They optimize existing machine models in terms of speed and accuracy.
- They debug, deploy, and maintain computer vision algorithms for different environments.
- They post-process the results and gather analytics.
- Develop technical documentation on the products created based on your CV solution to support end users/stakeholders on how the system works.
However, depending on the position and industry this might differ. In some positions, you might have to do more specific tasks or extend those ones (i.e, research on one specific domain and make some solution proposals around). But in general, the above describes the general tasks a computer vision engineer carries out.
Why to specialize in Computer Vision?
Images can contain very powerful and useful information. We deal with images on a daily basis. We unlock our phones using our faces, we record videos everywhere we go and this is just a simple use case. Analyzing images has led to significant advancements in various industries such as cancer treatment, satellite imagery, surveillance, retail, and many more.
In my opinion, there has been great progress in different branches of AI such as computer vision, NLP, Deep Learning, and Speech recognition. Computer vision deals with image and video data while NLP deals with text and general data-science deals with numerics. Choosing your direction definitely depends on your industry of interest. But computer vision has vast applications in the industry as compared to NLP and as a result higher impact. It is also worth noting that many scientists these days adapt themselves to work with various modalities as the research trend these days is to use a combination of various modalities for better accuracy and generalization.
Skills Required to Become a Computer Vision Engineer
One should possess the following skills to become a Computer Vision engineer.
- Strong object-oriented programming skills
- Sound knowledge of data structures
- Basic understanding of software development concepts and lifecycle
- Know-How of languages and libraries such as C++, Matlab, Python, SQL Server, Open CV, R, etc.
- Debugging skills
- Robust Software testing
- Good grasp of linear algebra practices like principal component analysis, dimensionality reduction, matrix factorization, matrix multiplication, linear transformation, etc.
- Ability to develop machine learning models and DL.
Image Source: Google
Job Market Opportunities
Computer Vision is a billion-dollar market and is growing by leaps and bounds over the past few years. The market is expected to hit a record of $17 billion by 2023.
The compensations and the job opportunities for Computer Vision Engineers vary across cities. Below is the table that shows the average salaries of computer vision engineers in various cities.
Roles and Responsibilities
The job roles and responsibilities in computer vision differ for each position in an organization. But some of the day-to-day tasks that a computer vision engineer performs are mentioned below.
- Reading the latest journals and research papers
- Developing Computer Vision Model
- Automating Computer Vision tasks such as parking vacancy detection, mask detection, etc
- Object Recognition
- Image Restoration
- Motion Analysis
- Reviewing code and collaborating with domain experts in Data Science and Machine Learning
Computer Vision Engineer Salary
The average salary of a computer vision engineer depends on his experience level, job role, country, and the company he/she works in. In the US, computer vision engineers earn up to 117k USD per annum on average.
The image is taken from PayScale.com
The salary of an entry-level computer vision engineer falls around 94k per annum. On the other hand, a senior-level computer vision engineer can earn more than 140k every year.
Libraries and Frameworks
Given below are the popular computer vision libraries and frameworks that one can use for various purposes depending on his needs.
- OpenCV – It is the most popular open-source computer vision library that has a few use cases like facial recognition, gesture recognition, object detection, human-computer interaction, etc. It can be used in different languages such as Java, C++, Python, etc., and supports Windows, Android, Linux, and macOS.
- TensorFlow – It facilitates the process of building Artificial Intelligence models. It has customized solutions such as TensorFlow Lite and TensorFlow.js.
- MATLAB – This paid programming platform helps with various computer vision-related tasks such as recognizing objects, performing stereo vision, calibrating cameras, detecting and tracking objects in video frames, etc.
- Keras – It is a Python-based open-source library that builds neural network models quickly. Some of its use cases include 3D image classification, image segmentation, handwriting recognition, etc.
- Pytorch – It is an open-source machine learning library that uses core Python concepts like structures, conditional loops, and classes to build complex architectures. We can use it for image classification, image segmentation, image estimation models, etc.
Computer Vision Techniques
Given below are some common computer vision techniques that make computers understand visual data from the real world.
Object Detection
It is the computer vision technique that refers to the detection and localization of objects using bounding boxes. It uses class-specific details to identify objects. The classes can be any object like animals, cars, etc on which the detection model is trained. Object Detection is often accompanied by Object Recognition (also known as Object Classification).
Image Source: Google
Face Recognition
It is a subpart of Object Recognition where the human face is the primary object to be detected. This technique not only performs detection, but also the recognition of the detected face. The facial recognition systems look for common features like eyes, lips, or nose and classify a face using these features.
Image Source: Google
Image Classification
It is the most popular computer vision technique used by beginners and experts. It classifies a group of images into a set of predefined classes using a set of sample images that have already been classified.
Image Segmentation
It is a computer vision technique that divides an image into subparts or sub-objects (segments). Segments are particular classes of objects identified by the neural network in an image. Image Segmentation demonstrates that the machine can recognize an object from the background or another object in the same image.
Some other computer vision techniques are Edge Detection, Image Restoration, Feature Matching, etc.
Best Books for Computer Vision
You always require a starting point whether you want to learn machine learning, deep learning, computer vision, or anything else. It is good to learn computer vision by doing some hands-on projects, but you will always require the basic knowledge of its underlying principles.
Here comes the need for some top-quality books that will help you understand the fundamentals, applications, pros, and cons of computer vision. Books are always good to read as they provide detailed explanations of every concept.
Below are some books that will help you comprehend the fundamentals of Computer Vision.
- Computer Vision: Models, Learning, and Inference, 2012. – It focuses on how to use training data to learn the relationships between the observed image data and aspects of the world that we wish to estimate.
- Computer Vision: Algorithms and Applications, 2010. – It explores various techniques used to analyze and interpret images.
- Multiple View Geometry in Computer Vision, 2004. – It focuses on geometric principles and algebraic representation of objects to understand the structure of the real world.
- Computer Vision: A Modern Approach, 2002. – It provides sufficient details for students to be able to build useful applications and gives a general view of the entire computer vision enterprise.
- Introductory Techniques for 3-D Computer Vision, 1998. – It provides an introduction to modern computer vision and focuses on a set of computational techniques for 3-D imaging.
- Practical Deep Learning for Cloud, Mobile, and Edge: Real-World AI & Computer-Vision Projects Using Python, Keras & TensorFlow, 2019. (** Featured as a learning resource on the official Keras website **) – it focuses on practical deep learning applications for mobile, cloud, browsers, and edge devices using a hands-on approach.
Beginner Friendly Online Courses
Online courses are pretty convenient since they allow you to follow a self-paced path and learn basic concepts while applying to practical assignments.
- Introduction to Computer Vision and Image Processing ( by Coursera, IBM): hands-on course for learning CV involving tech stacks such as Python, Pillow, and OpenCV for basic image processing and performing image classification and object detection. The interesting part: it also shows how to deploy your own model into the cloud.
- Introduction to Computer Vision (by Udacity, Georgia Tech): it shows fundamentals of image formation, camera imaging geometry, feature detection and matching, multiview geometry including stereo, motion estimation, tracking, and classification.
- Python for CV with OpenCV and DL (by Udemy, Jose Portilla): tech stack (NumPy, OpenCV, Tensorflow, Keras).
- Computer Vision Basics: alternative course for those ones who want to explore different non-python environments (MATLAB),
For a more advanced path, there are some interesting options, such as some NDs offered by Udacity (e.g, ND91 nano degree).
Research Papers
You should read research papers as they keep you up-to-date with the current technological advancements in computer vision (CV). Below are some papers that will help build a concrete understanding of computer vision concepts and applications.
- Forest Health Diagnosis from Aerial Images – It uses deep learning to look at the dead and living trees to analyze the health of the forests.
- Two-Stage Generative Adversarial Networks for Document Image Binarization – It focuses on taking in a scanned color document image, enhancing it, and binarizing it, so the document analysis tasks can be performed on it accurately.
- Reposing Humans by Warping 3D Features – It aims to repose the person in an input image to the target pose. The selfie-enthusiasts can easily make changes to their images to achieve the desired pose before posting.
Real-World Projects
Below are the best projects that depict the practical application of Computer Vision in the real world.
Detecting Pathologies in Ultrasound Images
It is challenging for pathologists to analyze large amounts of data in a day. This project uses OpenCV and Streamlit to detect pathologies in ultrasound images and evaluates issues in the human body like broken blood vessels, thinning skin, etc.
Image Source: Omdena
Automatic Colorization Using Deep Neural Networks
The aim of this project is to assign color information to grayscale images without any user intervention. It combines the use of various machine learning and deep learning networks for automatic colorization.
Image Source: Github
You can learn more about the project using this link.
Comparing Semantic and Instance Segmentation for Weed and Crop Detection
Agriculture is the backbone of a country’s economy where the growth of the products is directly related to the quantity and quality of crops. Farmers use various herbicides to protect their crops from weeds. However, modern developments in Artificial Intelligence can assist farmers in automating this process.
This computer vision project uses various deep learning segmentation techniques for detecting weeds and crops using drone imagery.
Image Source: Omdena
Text Recognition using OpenCV and Tesseract (OCR)
This project identifies each letter and converts it into text using OpenCV and Optical Character Recognition (OCR) on an image. Apps like Google Lens and PDF Scanner use OCR. Tesseract is an open-source application that recognizes text-based data in more than a hundred languages.
Image Source: Google
You can learn more about the project using this link.
Helping People with Visual Impairment to Easily Use Buses through Computer Vision
One of the biggest challenges faced by people with visual impairment is the ability to walk safely and accurately from one point to another point. This project aims to develop an automated Computer Vision pipeline to detect buses, bus lines, and empty seats.
Computer Vision Engineer Resume
Computer Vision engineers are in high demand as computer vision has become an important part of many companies’ business models. If you are looking for a job, follow the below tips to write a computer vision engineer resume and grab the hiring manager’s interest.
Identify and Include Relevant Keywords
When you apply for a computer vision engineer job, your resume is scanned by an application tracking system (ATS) for certain keywords. Your application might not pass the initial screening process if your resume does not include enough of the right terms.
You can make notes of keywords by looking at the job postings and including those terms in your resume. Some example keywords are:
- Python
- OpenCV
- Computer Vision
- Computer Vision Research
- Machine Learning
- Computer Graphics
- Image Recognition
- Deep Learning, etc.
Write Compelling Bullet Points
The more specific the bullet points are about your responsibilities and achievements, the better. For instance, consider these two bullet points.
- Developed a mobile application for image processing
- Developed a mobile application for image processing using Python to generate heap maps based on user interactions
The second bullet point is more specific as it provides more details about what you did in the project.
Showcase Your Technical Skills
As a computer vision engineer, you should be familiar with algorithms and computer vision concepts, and have experience with machine learning and image processing. Likewise, you should have a strong knowledge of programming languages like Python, C++, and systems like Windows and Linux.
Remember the Basics
You should keep the following basic rules in mind when drafting your resume.
- Use a summary
- Be concise
- Make it easy to scan
- Proofread your resume (check grammar mistakes and formatting errors)
Computer Vision Interview Questions
Below are some computer vision interview questions and answers.
- What is a digital image?
A digital image is a picture that is made up of small fragments called pixels. The pixels are composed of numerical components that represent their intensity and color codes.
- What are Computer Vision libraries?
A computer vision library is where a developer can store the functions and equations that are used to create neural networks. The computer uses them to learn and process data.
- What features does a computer vision neural network detect?
A computer vision neural network detects images such as landscapes, vehicles, household items, human faces, etc. Specific features include color, shape, patterns, and relative size.
- Name the machine learning algorithms you can use in OpenCV.
We can use a wide range of algorithms in OpenCV like convolutional neural networks, decision trees, random forest, K-nearest neighbor, gradient boosting trees, etc.
- What programming languages does Computer Vision support?
Computer Vision uses programming languages like Python, Java, C, C++, Prolog, etc.
Learn more through detailed examples of computer vision interview questions and answers.
Get Inspired
Get inspired by the magnitude of learnings and success stories of these amazing changemakers.
From Junior Machine Learning Engineer to Computer Vision Researcher
Image Source: Omdena
24 months ago Mahzad Khoshlessan started as a Junior Data Scientist at Omdena. She now works as a Research Scientist on 3D Computer Vision & Robot Perception at the Stevens Institute for Artificial Intelligence.
Read about her journey from joining her first Omdena AI Challenge to speaking at Deeplearning.ai in front of more than 1000 people, to becoming an AI Product Owner, and now considering entering an entrepreneurship program.
Interning at NVIDIA & overcoming barriers
Image Source: Omdena
Kennedy K. Wangari from Kenya shares his learnings on balancing courses and real-world projects, mindset, and securing an NVIDIA Internship.
Kennedy´s key lesson:
“The self-trust mindset: you’ve got to trust yourself, be courageous enough to follow your passion aggressively, and believe in your capabilities.”
From Omdena to a full-time offer at Microsoft
Image Source: Omdena
Kritika Rupauliha is a CS undergrad, currently in the 6th semester of her degree.
Kritika’s key lesson:
“Thriving happens in a diverse community of like-minded people.”
You can read more full article about How to Get a Data Science Job at Top Companies
Conclusion
Revolutions in visual AI technologies and Computer Vision drive the demand for engineering experts. Every computer vision engineer should have the ability to:
- Understand dataflow programming
- Use knowledge of computer vision libraries
- Develop deep learning architectures to solve problems
- Design platforms for image processing and visualization
We hope you enjoyed this article and got an idea of the computer vision basics, techniques, and practical applications of AI in the real world.
