When it comes to embedded computer vision, fractions of code acceleration are regarded as a huge success for programmers. Today, I’ll explain how to build a customized OpenCV for Raspberry Pi as one of the most famous single-board computers. By following these simple tips, you’ll experience a 2-3x faster OpenCV on your board.
Raspberry Pi 2 and higher versions have multi-core CPUs that support ARM NEON technology. Clearly, a code that benefits from these two options will run much faster than a simple bare code. The good news is that most OpenCV functions are parallelized on CPU and a limited number of them benefit from NEON C intrinsics. You can check this by digging into some of the source codes (e.g. KLT tracker implementation in opencv/src/modules/video/src/lkpyramid.cpp) and look for parallel_for_and if CV_NEONstatements.
The bad news is that, if you have previously built OpenCV on your board, it is most likely that your library doesn’t benefit from these options. You can simply check by running the following script:
if(cv::getNumThreads()==4 && cv::checkHardwareSupport(CV_CPU_NEON)==1)
std::cout << "OpenCV is optimized" << std::endl;
If it doesn't print the message, it means that your OpenCV doesn’t support multi-threading and/or NEON vectorization.
1- Uninstall your current OpenCV
To build a customized OpenCV with the mentioned capabilities, you must first delete your current version. If you had installed it via sudo apt install libopencv-dev, just run sudo apt purge libopencv-dev. But if you had installed it from source, you must first go to its build folder and then run:
sudo make uninstall
sudo rm -r *
To make sure that your Pi is clean from any OpenCV lib, run the following commands. If it prints, then there is something wrong with your uninstallation.
ls | grep -e libopencv
2 - Install an optimized OpenCV
Before building OpenCV itself, we must install some base dependencies:
mkdir OpenCV && cd OpenCV
wget -O opencv.zip https://github.com/opencv/opencv/archive/4.5.0.zip
wget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/4.5.0.zip
and finally, run:
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=RELEASE -DCMAKE_INSTALL_PREFIX=/usr/local -DWITH_TBB=ON -DWITH_LAPACK=ON -DENABLE_VFPV3=ON -DENABLE_NEON=ON -DBUILD_TESTS=OFF -DINSTALL_C_EXAMPLES=OFF -DINSTALL_PYTHON_EXAMPLES=OFF -DBUILD_EXAMPLES=OFF -DOPENCV_EXTRA_MODULES_PATH=~/OpenCV/opencv_contrib-4.5.0/modules ..
sudo make install
The flag –DWITH_TBB enables multi-threading, while the flag -DWITH_LAPACK enables faster matrix operations with OpenBLAS. The –DENABLE_VFPV3=ON and –DENABLE_NEON=ONflags enable OpenCV to use the NEON coprocessor for vectorization. Other flags just turn off building tests and example programs to reduce the overall OpenCV building time.
After building, you have an OpenCV that certainly works faster. The following figure shows the boost-up of some of the well-known functions.
If you are using a 64bit OS on your Pi, then don't set –DENABLE_VFPV3=ON and –DENABLE_NEON=ON. It throws an error. CMake itself will find the co-processors automatically (tested with OpenCV 4.2.0 on Ubuntu Mate 18).
Although the documents suggest more threading frameworks like OpenMP, pthreads, Concurrency, and GCD, I didn’t experience a multi-threaded OpenCV after setting –DWITH_OPENMP=ON.
You can also use libjpeg-turbo instead of OpenCV’s default libjpeg library. It uses NEON instructions to boost up Jpeg reading and writing. If you are interested, you must first build it by:
wget https://github.com/libjpeg-turbo/libjpeg-turbo/archive/1.5.0.tar.gz -O libjpeg-turbo.tar.gz
tar xvf libjpeg-turbo.tar.gz
sudo make install
and then add the following statements to your CMake command:
The instructions mentioned in this post are not limited to Raspberry Pi. They are applicable to all SBCs equipped with multi-core or newer processors that support Neon technology, including Odroid XU4, BeagleBoard X15, Nvidia Jetson series, etc.