Contents


Introduction

Concatenating images aka creating a larger image by placing multiple smaller images next to each other is a common thing to do. In general, this is a straight forward process using OpenCV however, there are a couple of different ways to do and not all are applicable in all cases. When prototyping in Python knowing that the final version is going to be programmed using C++, then makes little sense to use stitching functions that are specific to the Python API.

NB!: Image stitching usually implies creating one picture out of many smaller ones with transformations applied to create an error-free panoramic image.

The fundamental requirement for concatenating images is that they are of the same type and compatible shape. Concatenating a grayscale and a BGR images requires converting the grayscale image to BGR first. The axis of concatenation needs to be of the same size as well.

Concatenating cv::Mat

When concatenating cv::Mat or cv::UMat, then there are basically two options:

  • the easy one (cv::hconcat/cv::vconcat)
  • the manual memory allocation and copy approach (copyTo)

cv::hconcat and cv::vconcat can be used in different ways. If only two cv::Mat should be concatenated, then cv::hconcat(img_0, img_1, img_result) does the job. When concatenating multiple images, then an array or vector containing input images for concatenation can be used:

// C++

#include <opencv2/opencv.hpp>
#include <vector>

int main()
{
    cv::Size mat_size(100,100);
    cv::Mat blue(mat_size, CV_8UC3, cv::Scalar(255,0,0));
    cv::Mat green(mat_size, CV_8UC3, cv::Scalar(0,255,0));
    cv::Mat red(mat_size, CV_8UC3, cv::Scalar(0,0,255));
    cv::Mat h_concat_0, h_concat_1, h_concat_2;
    cv::Mat v_concat;
    cv::Mat h_v_concat;

    std::vector<cv::Mat> imgs_for_concat_0 = {blue, green, red};
    std::vector<cv::Mat> imgs_for_concat_1 = {red, blue, green};
    std::vector<cv::Mat> imgs_for_concat_2 = {green, red ,blue};

    cv::hconcat(imgs_for_concat_0, h_concat_0);
    cv::hconcat(imgs_for_concat_1, h_concat_1);
    cv::hconcat(imgs_for_concat_2, h_concat_2);
    cv::vconcat(imgs_for_concat_0, v_concat);

    std::vector<cv::Mat> imgs_for_concat_3 = {h_concat_0, h_concat_1, h_concat_2};
    cv::vconcat(imgs_for_concat_3, h_v_concat);
}

The python code looks identical. Both tuple and list can be used:

# Python

import cv2

# ...

h_concat = cv2.hconcat([blue,green,red])
v_concat = cv2.vconcat((blue,green,red))

cv::hconcat concatenates images horizontally:

OpenCV hconcat


cv::vconcat concatenates images vertically:

OpenCV vconcat


And of course they can be combined:

OpenCV h_vconcat


The second approach is a bit more fundamental. A new cv::Mat of desired output size could be pre-allocated and all single images are copied into their correct position in the output image using copyTo.

cv::Size mat_size(100,100);
cv::Size h_concat_size(300,100);
cv::Size v_concat_size(100,300);
cv::Mat blue(mat_size, CV_8UC3, cv::Scalar(255,0,0));
cv::Mat green(mat_size, CV_8UC3, cv::Scalar(0,255,0));
cv::Mat red(mat_size, CV_8UC3, cv::Scalar(0,0,255));
std::vector<cv::Mat> imgs_for_concat_0 = {blue, green, red};

// initialize new mats
cv::Mat h_concat(h_concat_size, CV_8UC3);
cv::Mat v_concat(v_concat_size, CV_8UC3);
int start_col = 0;
int start_row = 0;
int curr_cols;
int curr_rows;

// hconcat
for (size_t i=0; i < imgs_for_concat_0.size(); ++i)
{
    curr_cols = imgs_for_concat_0[i].cols;
    curr_rows = imgs_for_concat_0[i].rows;
    imgs_for_concat_0[i].copyTo(h_concat(cv::Rect(start_col, start_row, curr_cols, curr_rows)));
    start_col += curr_cols;
}

// vconcat
start_col = 0;
start_row = 0;
for (size_t i=0; i < imgs_for_concat_0.size(); ++i)
{
    curr_cols = imgs_for_concat_0[i].cols;
    curr_rows = imgs_for_concat_0[i].rows;
    imgs_for_concat_0[i].copyTo(v_concat_size(cv::Rect(start_col, start_row, curr_cols, curr_rows)));
    start_row += curr_rows;
}

In theory this approach has the advantage that images of incompatible shapes do not have to be reshaped first. That however could be considered bad practice ;).

Concatenating cv::cuda::GpuMat

There exists no such equivalent to cv::hconcat/cv::vconcat for the cv::cuda namespace/capabilities. Therefore, we are limited to the.copyTo approach which works exactly as shown above just with cv::cuda::GpuMat instead of cv::cuda::Mat.

Concatenating np.array

When using the OpenCV’s Python API, then images (cv::Mat) are NumPy arrays (np.ndarray). Besides the options above, NumPy offers three options:

  • np.hstack((img_0,img1)) or np.concatenate((img_0,img1), axis=1)
  • np.vstack((img_0,img1)) or np.concatenate((img_0,img1), axis=0)