We all know photographs mean us a lot. They keep our memories alive for life time. Therefore, when we click a picture, we capture a moment for eternity! Therefore, a single photograph can take you down memory lane and remind you of people, places and the beautiful memories you have collected over time. But what if those captured photographs are blurred or degraded due to weather conditions such as haze or rain? It would be disappointing, wouldn’t it? One cannot definitely go back in time to correct those photographs but now Dr. Rajagopalan’s team in IIT Madras can digitally restore these pictures and preserve your fond memories.
The field of photography has come a long way from being a tool to preserve memories to being used now for surveillance, drone flying, futuristic autonomous driving systems and various other domains. These applications require that the images are clean and not blurry or damaged. However, weather conditions pose a major challenge in getting clean images. In places such as hilly areas or where rainfall is common, surveillance cameras which are placed in cities cannot be relied upon for monitoring as they cannot provide clean camera footage.
Blurred 1 
Blurred 2 
Cleaned 1 
Cleaned 2
“Bad weather in the form of rain and haze causes significant loss of image quality. The presence of rain-drops on camera lenses is a related problem that poses its own set of challenges. These effects not only impact human visibility but can also adversely affect the performance of computer vision systems meant for autonomous driving, drone imaging, and surveillance, to name a few. These degradations have high spatial variability due to non-uniform depth variations in the haze, drop sizes and their locations in raindrops, and rain streak directions and locations,” says Dr A.N Rajagopalan, who holds the Sterlite Technologies Chair Professor position in the Department of Electrical Engineering at IIT Madras, while elaborating on the issue.
Dr Rajagopalan’s Image Processing and Computer Vision Lab at IIT Madras is utilizing the power of artificial neural networks to restore degraded images. In their recent work which has been published in the prestigious IEEE Journal of Selected Topics in Signal Processing, the team has come up with an approach using neural networks that can be utilized to clean images affected by rain-streaks, raindrops, haze and motion blur. They found that identifying the degraded portions as well as cleaning the image was tricky for a single neural network. Therefore, they decided to split the task in two stages. In the first stage, one network was trained to localize the degraded part and in the second step, a subsequent network harnessed this information to restore the image.
“We have developed a deep learning network that decomposes the restoration task into two stages: degradation localization and degraded region-guided restoration. Our premise is to use the auxiliary task of degradation mask prediction to guide the restoration process. We demonstrate that solving this auxiliary task injects crucial localizing ability in network layers. We transfer this ability to the main restoration network using attentive knowledge-distillation and focus on the refinement of degraded regions by exploiting this additional knowledge,” explains Dr. Rajagopalan.
The team utilized publicly available datasets of rain streak, haze, raindrop and motion blur to test their model by employing knowledge distillation for image restoration coupled with prediction of degraded locations. This strategy used by the team was found to outperform competing methods.
L-R (Maitreya Suin , Prof.A N Rajagopalan, Kuldeep Purohit)
“The authors show the applicability of the degradation mask prediction. They have a one-size-fits-all framework to handle some rather compelling and practical effects of rain-streak removal, haze removal, raindrop removal and motion blur removal: applications that have had prior work specifically focus on them. The spatially-varying restoration of degraded images, guided by the degradation mask is the big take-home point as a generic direction of viable research in the area” comments Prof. Sumantra Dutta Roy from IIT Delhi who works in a similar area. The team led by Dr Rajagopalan included Maitreya Suin and Kuldeep Purohit from IIT Madras.
Article by Article by Aditi Jain
Here is the original link to the scientific paper:
https://ieeexplore.ieee.org/document/9288928
