Image Similarity using Feature-based Similarity Index Matrix with Python

Have you ever wondered how Google Lens, Google Image Search, and Bing Image Search work?

All these tools have a few techniques in common, the technique they use is known as Image Similarity; it is a process in which multiple images are compared and most similar photos based on rank are shown.

Image Similarity can be achieved in various ways using Siamese Networks, Structural Similarity, Feature Similarity, etc.

These techniques take vectorized forms of images and compare their feature vectors to derive inference.

In Today’s blog, we will create a Pipeline to identify whether two images are similar or not using the Feature-based Similarity Index Matrix.
Feature-based Similarity Index Matrix maps the features and measures the similarities between two images.

To describe FSIM we need to describe two criteria more clearly. They are Phase Congruency (PC) and Gradient Magnitude (GM).

Phase Congruency (PC): A new method for detecting image features is phase congruency. One of the important characteristics of phase congruency is that it is invariant to light variation in images. Besides, it is also able to detect more interesting features. It stresses the features of the image in the domain frequency. Phase congruency is invariant to contrast.

Gradient magnitude (GM): Image gradient computation is a very traditional topic in digital image processing. Convolution masks are used to express the operators of the gradient. There are many convolutional masks to measure the gradients. If f(x) is an image and Gx, Gy of its horizontal and vertical gradients, respectively. Then the gradient magnitude of f(x) can be defined as

Let two images be f1 (test image) and f2 (reference image) and their phase congruency can be denoted by PC1 and PC2, respectively. The Phase Congruency (PC) maps were extracted from two images f1 and f2 and the Magnitude Gradient (GM) maps G1 and G2 extracted from the two images too. FSIM can be defined and calculated based on PC1, PC2, G1, and G2. At first, we can calculate the similarity of these two images as

We will also work on creating a Live Demo using Gradio and deploy it on TrueFoundry.

Let’s begin

Importing Libraries

import cv2
import numpy as np
import gradio as gr
import phasepack.phasecong as pc

Creating our Pipeline

Here, we will a series of functions that will help us to implement FSIM (see original Paper)

1. Helper Functions

These functions will help us to measure the gradient magnitude.

Gradient magnitude describes the local rate of change in the scalar field.

2. FSIM Function

This is our primary function which implements the Feature-based Similarity Index Matrix. It takes two images and PC and GM Constants and will give FSIM Score

Building Gradio App

We are going to create our Gradio App and run it on port 8080

Test Drive

Running our App

Voila! it runs, Now we will work on deploying this.

Deployment

We are going to use TrueFoundry for our deployment

Logging into TrueFoundry

Heading to Deployment Section

1) Creating a new deployment

2) Select the Service option and Workspace name

3) Fill out properties and submit

4) Deploying

5) Successful Deployment

6) Final Thoughts

After the deployment, you will be able to use the Gradio App.

The app is deployed here: https://image-similarity-demo-projects.tfy-ctl-euwe1-production.production.truefoundry.com/

Video

Code

The above code is also present in my Repository

References:

1. TrueFoundry: https://truefoundry.com/
2. TrueFoundry App: https://app.truefoundry.com/
3. TrueFoundry Docs: http://docs.truefoundry.com/
4. Code: https://github.com/d4rk-lucif3r/Feature-based-Similarity-Index
5. FSIM: https://www.scirp.org/journal/paperinformation.aspx?paperid=90911#:~:text=Feature%20Similarity%20Index%20Method%20maps,image%20features%20is%20phase%20congruency.