這篇博客將介紹如何使用OpenCV和深度學(xué)習(xí)應(yīng)用全面嵌套的邊緣檢測(cè)。并將對(duì)圖像和視頻流應(yīng)用全面嵌套邊緣檢測(cè)，然后將結(jié)果與OpenCV的標(biāo)準(zhǔn)Canny邊緣檢測(cè)器進(jìn)行比較。
1. 效果圖
憤怒的小鳥——原始圖 VS Canny邊緣檢測(cè)圖 VS HED邊緣檢測(cè)圖
花朵——原始圖 VS Canny邊緣檢測(cè)圖 VS HED邊緣檢測(cè)圖
視頻效果圖GIF 如下
2. 全面嵌套邊緣檢測(cè)與Canny邊緣檢測(cè)
2.1 Hed與Canny邊緣檢測(cè)對(duì)比
Holistically-Nested Edge Detection (HED) 全面嵌套邊緣檢測(cè)
Canny Edge Detection Canny邊緣檢測(cè)
OpenCV 利用Canny邊緣檢測(cè)能夠找到圖像中對(duì)象的邊界。但是Canny邊緣檢測(cè)器存在一些問(wèn)題，即：
- 需要手動(dòng)驗(yàn)證（將下部和上值設(shè)置為滯后閾值，是一種需要實(shí)驗(yàn)和視覺(jué)驗(yàn)證的手動(dòng)過(guò)程）；
- 不具備通用性（對(duì)不同照明條件下捕獲的相同圖像，適用于一個(gè)圖像，卻不適用于另一個(gè)圖像）；
- 通常需要許多預(yù)處理步驟（即轉(zhuǎn)換為灰度，模糊/平滑等），以獲得良好的邊緣圖。
整體嵌套邊緣檢測(cè)（HED）試圖通過(guò)端到端深神經(jīng)網(wǎng)絡(luò)解決Canny邊緣檢測(cè)器的局限性。
該網(wǎng)絡(luò)接受RGB圖像作為輸入，然后將邊緣圖作為輸出產(chǎn)生。而且通過(guò)HED產(chǎn)生的邊緣圖在圖像中很好的保留了對(duì)象邊界。
2.2. 項(xiàng)目結(jié)構(gòu)
2.3 deploy.prototxt, hed_pretrained_bsds.caffemodel下載
執(zhí)行代碼的關(guān)鍵是獲取deploy.prototxt, hed_pretrained_bsds.caffemodel
https://github.com/opencv/opencv/blob/master/samples/dnn/edge_detection.py
https://github.com/seminar2012/hed
- This sample shows how to define custom OpenCV deep learning layers in Python.
- Holistically-Nested Edge Detection (https://arxiv.org/abs/1504.06375) neural network is used as an example model.
- Find a pre-trained model at https://github.com/s9xie/hed. We provide the pretrained model and training/testing code for the edge detection framework Holistically-Nested Edge Detection (HED).
- Please see the Arxiv or ICCV paper for technical details. The pretrained model (fusion-output) gives ODS=.790 and OIS=.808 result on BSDS benchmark dataset.
- Download the pretrained model (56MB) from (http://vcl.ucsd.edu/hed/hed_pretrained_bsds.caffemodel) and place it in examples/hed/ folder.
3. 源碼
3.1 對(duì)圖像進(jìn)行HED檢測(cè)
# USAGE
# python detect_edges_image.py --edge-detector hed_model --image images/bird.jpg
# 導(dǎo)入必要的包
import argparse
import cv2
import os
import imutils
# 構(gòu)建命令行參數(shù)及解析
# --edge-detector Holistically-Nested Edge Detection檢測(cè)器模型路徑
# --image 圖片路徑
ap = argparse.ArgumentParser()
ap.add_argument("-d", "--edge-detector", type=str, required=True,
help="path to OpenCV's deep learning edge detector")
ap.add_argument("-i", "--image", type=str, required=True,
help="path to input image")
args = vars(ap.parse_args())
class CropLayer(object):
def __init__(self, params, blobs):
# 初始化剪切區(qū)域開始和結(jié)束點(diǎn)的坐標(biāo)
self.xstart = 0
self.ystart = 0
self.xend = 0
self.yend = 0
# 計(jì)算輸入圖像的體積
def getMemoryShapes(self, inputs):
# 剪切類將接收倆個(gè)參數(shù)
# 剪切第一個(gè)輸入blob以匹配第二個(gè)blob，保持批次和通道數(shù)
# 輸出輸入容積的形狀及目標(biāo)形狀
# 提取批量大小及通道數(shù)
# 分別提取目標(biāo)形狀的高和寬
(inputShape, targetShape) = (inputs[0], inputs[1])
(batchSize, numChannels) = (inputShape[0], inputShape[1])
(H, W) = (targetShape[2], targetShape[3])
# 計(jì)算開始和結(jié)束剪切坐標(biāo)的值
self.xstart = int((inputShape[3] - targetShape[3]) // 2)
self.ystart = int((inputShape[2] - targetShape[2]) // 2)
self.xend = self.xstart + W
self.yend = self.ystart + H
# 返回體積，接下來(lái)進(jìn)行實(shí)際裁剪
return [[batchSize, numChannels, H, W]]
def forward(self, inputs):
return [inputs[0][:, :, self.ystart:self.yend, self.xstart:self.xend]]
# 從磁盤加載序列化的邊緣檢測(cè)器模型
print("[INFO] loading edge detector...")
protoPath = os.path.sep.join([args["edge_detector"],
"deploy.prototxt"])
modelPath = os.path.sep.join([args["edge_detector"],
"hed_pretrained_bsds.caffemodel"])
net = cv2.dnn.readNetFromCaffe(protoPath, modelPath)
# 綁定剪裁類到模型
cv2.dnn_registerLayer("Crop", CropLayer)
# 加載輸入圖像，獲取其維度
image = cv2.imread(args["image"])
image = imutils.resize(image, width=400)
(H, W) = image.shape[:2]
# 轉(zhuǎn)換圖像為灰度圖，高斯平滑，執(zhí)行Canny邊緣檢測(cè)
print("[INFO] performing Canny edge detection...")
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blurred = cv2.GaussianBlur(gray, (5, 5), 0)
canny = cv2.Canny(blurred, 30, 150)
# 根據(jù)輸入圖像為全面的嵌套邊緣檢測(cè)器（Holistically-Nested Edge Detector）構(gòu)建一個(gè)輸出blob
blob = cv2.dnn.blobFromImage(image, scalefactor=1.0, size=(W, H),
mean=(104.00698793, 116.66876762, 122.67891434),
swapRB=False, crop=False)
# # 設(shè)置blob作為網(wǎng)絡(luò)的輸入并執(zhí)行算法以計(jì)算邊緣圖
print("[INFO] performing holistically-nested edge detection...")
net.setInput(blob)
hed = net.forward()
# 調(diào)整輸出為原始圖像尺寸的大小
hed = cv2.resize(hed[0, 0], (W, H))
# 將圖像像素縮回到范圍[0,255]并確保類型為“UINT8”
hed = (255 * hed).astype("uint8")
# 展示HED邊緣檢測(cè)的結(jié)果及Canny邊緣檢測(cè)的結(jié)果
cv2.imshow("Input", image)
cv2.imshow("Canny", canny)
cv2.imshow("HED", hed)
cv2.waitKey(0)
3.2 對(duì)視頻進(jìn)行HED檢測(cè)
# USAGE 默認(rèn)使用電腦自帶的攝像頭
# python detect_edges_video.py --edge-detector hed_model
# 使用視頻文件流
# python detect_edges_video.py --edge-detector hed_model --input xl.mp4
# 導(dǎo)入必要的包
from imutils.video import VideoStream
import argparse
import imutils
import time # 此模塊允許放置睡眠命令以允許視頻流建立和“熱身”。
import cv2
import os
# 構(gòu)建命令行參數(shù)及解析
# --edge-detector Holistically-Nested Edge Detection檢測(cè)器模型路徑
# --input 視頻源：網(wǎng)絡(luò)攝像頭，視頻文件或其他源。
ap = argparse.ArgumentParser()
ap.add_argument("-d", "--edge-detector", type=str, required=True,
help="path to OpenCV's deep learning edge detector")
ap.add_argument("-i", "--input", type=str,
help="path to optional input video (webcam will be used otherwise)")
args = vars(ap.parse_args())
class CropLayer(object):
def __init__(self, params, blobs):
# 初始化剪切區(qū)域開始和結(jié)束點(diǎn)的坐標(biāo)
self.xstart = 0
self.ystart = 0
self.xend = 0
self.yend = 0
# 計(jì)算輸入圖像的體積
def getMemoryShapes(self, inputs):
# 剪切類將接收倆個(gè)參數(shù)
# 剪切第一個(gè)輸入blob以匹配第二個(gè)blob，保持批次和通道數(shù)
# 輸出輸入容積的形狀及目標(biāo)形狀
# 提取批量大小及通道數(shù)
# 分別提取目標(biāo)形狀的高和寬
(inputShape, targetShape) = (inputs[0], inputs[1])
(batchSize, numChannels) = (inputShape[0], inputShape[1])
(H, W) = (targetShape[2], targetShape[3])
# 計(jì)算開始和結(jié)束剪切坐標(biāo)的值
self.xstart = int((inputShape[3] - targetShape[3]) // 2)
self.ystart = int((inputShape[2] - targetShape[2]) // 2)
self.xend = self.xstart + W
self.yend = self.ystart + H
# 返回體積，接下來(lái)進(jìn)行實(shí)際裁剪
return [[batchSize, numChannels, H, W]]
def forward(self, inputs):
# 使用派生（x，y）-oordinate來(lái)執(zhí)行裁剪
return [inputs[0][:, :, self.ystart:self.yend, self.xstart:self.xend]]
# 初始化視頻流，腳本將動(dòng)態(tài)選取使用視頻文件流還是網(wǎng)絡(luò)攝像頭流
webcam = not args.get("input", False)
# 如果未提供視頻文件路徑，則使用電腦自帶攝像頭
if webcam:
print("[INFO] starting video stream...")
vs = VideoStream(src=0).start()
time.sleep(2.0)
# 否則，獲取視頻文件流指針
else:
print("[INFO] opening video file...")
vs = cv2.VideoCapture(args["input"])
# 從磁盤加載序列化的HED檢測(cè)器模型
print("[INFO] loading edge detector...")
protoPath = os.path.sep.join([args["edge_detector"],
"deploy.prototxt"])
modelPath = os.path.sep.join([args["edge_detector"],
"hed_pretrained_bsds.caffemodel"])
net = cv2.dnn.readNetFromCaffe(protoPath, modelPath)
# 將剪裁類注冊(cè)到模型
cv2.dnn_registerLayer("Crop", CropLayer)
# 遍歷視頻流的幀
while True:
# 獲取每一幀，如果使用網(wǎng)絡(luò)攝像頭，獲取下一幀
frame = vs.read()
frame = frame if webcam else frame[1]
# 如果在處理視頻文件流，沒(méi)有獲取到幀則代表已經(jīng)到了文件尾部，則跳出循環(huán)
if not webcam and frame is None:
break
# 等比例縮放幀為寬度500，并獲取其維度
frame = imutils.resize(frame, width=300)
(H, W) = frame.shape[:2]
# 轉(zhuǎn)換灰度圖，高斯模糊并執(zhí)行Canny邊緣檢測(cè)
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
blurred = cv2.GaussianBlur(gray, (5, 5), 0)
canny = cv2.Canny(blurred, 30, 150)
# 為HED邊緣檢測(cè)器構(gòu)建輸入幀的blob，設(shè)置blob，并執(zhí)行檢測(cè)以計(jì)算邊緣圖
blob = cv2.dnn.blobFromImage(frame, scalefactor=1.0, size=(W, H),
mean=(104.00698793, 116.66876762, 122.67891434),
swapRB=False, crop=False)
net.setInput(blob)
hed = net.forward()
hed = cv2.resize(hed[0, 0], (W, H))
hed = (255 * hed).astype("uint8")
# 展示Canny、HED的檢測(cè)結(jié)果
cv2.imshow("Frame", frame)
cv2.imshow("Canny", canny)
cv2.imshow("HED", hed)
key = cv2.waitKey(1) 0xFF
# 按下‘q'鍵表示退出循環(huán)
if key == ord("q"):
break
# 如果在使用網(wǎng)絡(luò)攝像頭流，則終止相機(jī)視頻流
if webcam:
vs.stop()
# 否則，釋放視頻文件流指針
else:
vs.release()
# 關(guān)閉所有打開的window
cv2.destroyAllWindows()
參考
https://www.pyimagesearch.com/2019/03/04/holistically-nested-edge-detection-with-opencv-and-deep-learning/
到此這篇關(guān)于使用Python中OpenCV和深度學(xué)習(xí)進(jìn)行全面嵌套邊緣檢測(cè)的文章就介紹到這了,更多相關(guān)OpenCV和深度學(xué)習(xí)全面嵌套邊緣檢測(cè)內(nèi)容請(qǐng)搜索腳本之家以前的文章或繼續(xù)瀏覽下面的相關(guān)文章希望大家以后多多支持腳本之家！
您可能感興趣的文章:- Python OpenCV實(shí)現(xiàn)邊緣檢測(cè)
- OpenCV中Canny邊緣檢測(cè)的實(shí)現(xiàn)
- Python實(shí)現(xiàn)Opencv cv2.Canny()邊緣檢測(cè)
- OpenCV實(shí)現(xiàn)灰度、高斯模糊、邊緣檢測(cè)的示例
- Python使用Opencv實(shí)現(xiàn)邊緣檢測(cè)以及輪廓檢測(cè)的實(shí)現(xiàn)
- 如何利用Python 進(jìn)行邊緣檢測(cè)
- python實(shí)現(xiàn)canny邊緣檢測(cè)
- OpenCV半小時(shí)掌握基本操作之邊緣檢測(cè)
