As photovoltaic (PV) panels are installed outdoors, they are exposed to harsh environments that can degrade their performance. PV cells can be coated with a protective material to protect them from the environment. However, the coated area has relatively small temperature differences, obtaining a sufficient database for training is difficult, and detection in low-resolution thermal images is complicated. This paper proposes a method for detecting the relative temperature difference on PV panels and a method for accumulating detection results within consecutive thermal images. To verify the performance of the proposed method, we installed PV panels coated with three different materials. Subsequently, 60 infrared (IR) thermal and visible images were acquired using an IR thermal imaging camera mounted on the drone. When more than 16 out of 60 results were accumulated, the highest performance was achieved with an F1 score of 0.7178. This case study demonstrated that even low-resolution thermal images can be acquired continuously to detect areas with small temperature differences without applying machine learning, which requires a large database.