【Deep Learning OCR Series · 8】 Cikakken bayani game da algorithms na gano rubutu

## Gabatarwa Binciken rubutu shine mataki na farko kuma mai mahimmanci a cikin tsarin OCR. Manufarta ita ce gano yankunan rubutu a cikin hotuna, samar da daidaitattun bayanai don fahimtar rubutu na gaba. Tare da ci gaban fasahar ilmantarwa mai zurfi, algorithms na gano rubutu sun sami babban canji daga hanyoyin gargajiya zuwa hanyoyin ilmantarwa masu zurfi. Wannan labarin zai shiga cikin algorithms na gano rubutu na yau da kullun, gami da EAST, DBNet, PSENet, da sauransu, da kuma nazarin ka'idodin ƙirar su, fasalin fasaha, da yanayin aikace-aikace. ## Matsalolin da ke tattare da binciken rubutu ### Yanayin rikitarwa Binciken rubutu a cikin duniyar gaske yana fuskantar matsaloli da yawa: ** Geometric Deformation **: - Perspective Distortion: Geometric distortion due to shooting angles - Curved Text: Rubutu a kan lankwasawa saman kamar allunan talla da kwalabe - Juya Rubutu: Shirya rubutu a kowane kusurwa - Scale bambance-bambance: Bambance-bambance a rubutu size saboda daban-daban nesa ** Abubuwan da ke sha'awar gani **: - Hadaddun baya: Rubutun da baya suna kama da launi da rubutu - Canje-canje na haske: Tasiri kamar haske mai haske, inuwa, tunani, da dai sauransu - Blur da amo: motsi blur, mayar da hankali mara kyau, hayaniyar hoto - Matsalolin Occlusion: Wasu rubutun suna ɓoye ta wasu abubuwa ** Bambancin Rubutu **: - Multilingual Mix: Sinci, Turanci, Larabci, da sauran harsuna daban-daban - Bambance-bambance na font: bugawa, rubutun hannu, da rubutun fasaha - Tsari mai kauri: Ƙananan layin layi da haruffa masu ƙarfi - Multi-directionality: A kwance, a tsaye, karkatarwa, da dai sauransu a cikin hanyoyi daban-daban ### Bukatun Fasaha ** Abubuwan da ake buƙata **: - High tunawa: ba za a iya rasa rubutu yankunan - High daidaito: Guji misdetecting non-rubutu yankunan - Border Precision: Gano iyakokin rubutu daidai ** Abubuwan da ake buƙata **: - Real-Time Processing: Mobile apps bukatar zama amsawa - Iyakokin Albarkatu: Iyakokin Kwamfuta da ƙwaƙwalwar ajiya don na'urorin hannu - Batch Processing: Ingantaccen sarrafa manyan takardu ## Hanyoyin Gano Rubutu na Gargajiya ### Tsarin T **MSER(Maximally Stable Extremal Regions)**: - Ka'ida: Nemi tsayayyen yankuna a cikin hoton - Abũbuwan amfãni: Robust zuwa hasken canje-canje, high lissafi ingancin - Cons: Sensitive to complex backgrounds, difficult to handle low-contrast text ** SWT (Stroke Width Transform) **: - Ka'ida: Rubutun gano daidaito dangane da faɗin bugun jini - Amfani: Adaptable to font canje-canje - Disadvantages: Sensitive to parameters, sensitive to noise ### Tsarin T ** Multi-Scale Detection **: - Yi amfani da tagogi masu girma dabam dabam - Ya haɗu da siffofin hannu kamar HOG da LBP - Yi amfani da rarrabuwa kamar SVMs don sanin ko rubutu ne ko a'a * Binciken fa'idodi da rashin amfani: Na gaba: Abu mai sauƙi kuma mai sauƙin aiwatarwa - Cons: High computational complexity, da wuya a rike arbitrary-siffar rubutu ## Zurfin ilmantarwa rubutu gano algorithms ### Cikakken bayani game da Tsarin EAST (Ingantaccen da Daidaitaccen Scene Text Detector) ingantaccen algorithm ne na gano rubutu wanda aka gabatar a cikin 2017. ** Core Idea **: - Hasashen kai tsaye na yankunan rubutu: Guji matakai masu rikitarwa bayan sarrafawa - Multi-sikelin siffofin fusion: Haɗa matakai daban-daban na bayanin fasali - Hasashen Geometry: Yana tallafawa juyawa rectangle da ganowa quad ** Tsarin cibiyar sadarwa **: **1. Cibiyar sadarwar haɓaka fasali **: - Cibiyar sadarwar kashin baya: PVANet ko ResNet - Feature Pyramid: Ya haɗa da siffofi da yawa - Upsampling: Maido da ƙudurin taswirar fasalin **2. Tsinkaya reshe **: - Score graph: Hasashen yiwuwar cewa kowane pixel yana cikin rubutun - Geometry: Hasashen sigogin geometric na yankin rubutu ** Hasashen Geometric Parameter **: ** Yanayin juyawa na rectangle **: - 4 Nesa dabi'u: Nisa zuwa ɓangarori huɗu na rectangle - 1 kusurwa darajar: kusurwar juyawa na rectangle ** Yanayin Quad **: - 8 daidaita dabi'u: daidaita daidaitawa na vertices huɗu na quadrant ** Tsarin Aikin Asarar **: ** Asarar Classified **: Cross-entropy asarar ta amfani da Class Equilibrium: L_cls = -β * y * log (ŷ) - (1-β) * (1-y) * log (1-ŷ) ** Regression Loss **: Yin amfani da haɗuwa da asarar IoU da sauƙaƙe asarar L1: L_geo = L_IoU + λ * L_smooth_L1 ** Post-Processing **: - Thresholding: Generate candidate regions based on the score graph - NMS: Non-maximal inhibition of removal of repeat assays - Geometric constraints: Tace fitar da irrational geometries ### Cikakken bayani game da DBNet algorithm DBNet (Differentiable Binarization Network) algorithm ne na gano rubutu na ainihi wanda aka gabatar a cikin 2020. ** Core Innovation **: - Bambancin Binarization: Haɗa tsarin binarization a cikin cibiyar sadarwa - Daidaitattun ƙofofi: Ana koyon mafi kyawun ƙofofi ga kowane pixel - Sauƙaƙe Post-Processing: Rage rikitarwa na post-processing ** Tsarin cibiyar sadarwa **: **1. Backbone Network**: - ResNet-18/50: Siffar cirewa - FPN: Feature Pyramid Network - Upsampling: Revert to 1/4 ƙuduri na asalin hoto **2. Shugaban tsinkaye **: - Probability Graph P: Text area probability - Threshold Graph T: Adaptive binarization threshold - Binary Figure B: Sakamakon binarization na ƙarshe ** Bambancin binarization **: ** Standard binarization **: B = 1 idan P > T else 0 * Bambancin Bambancin **: B = 1 / (1 + exp (-k * (P-T))) inda K shine maɓallin haɓakawa, yana sa aikin ya fi kusa da aikin mataki. ** Aikin Asarar **: ** Jimlar asara **: L = L_cls + α * L_dis + β * L_thresh - L_cls: Asarar rarrabuwa (binary cross-entropy) - L_dis: Asarar nesa (Asarar Dice) - L_thresh: Threshold loss (L1 loss) ** Dabarun horo **: - Wuyar Samfurin Ma'adinai: Mayar da hankali kan pixels waɗanda suke da wahalar rarrabawa - Haɓaka bayanai: juyawa, haɓakawa, canjin launi - Multi-sikelin horo: Inganta daidaitawa zuwa girma daban-daban na rubutu ### Cikakken bayani game da PSENet algorithm PSENet (Progressive Scale Expansion Network) an tsara shi musamman don gano rubutun da ba a san shi ba. ** Core Idea **: - Ci gaba da fadada: Sannu a hankali fadada daga ƙananan kernels zuwa cikakken rubutu yankuna - Multiscale Kernels: Samar da rubutu cores na daban-daban masu girma dabam - Pixel Aggregation: Sake gina rubutun rubutu ta hanyar haɗuwa da matakin pixel. ** Tsarin cibiyar sadarwa **: **1. Siffar cirewa **: - ResNet backbone cibiyar sadarwar - FPN feature fusion - Rassa masu hasashe da yawa **2. Hasashen Multiscale **: Generate n segmentation plots at different scales: - S1: Ƙananan kernel (yankin cibiyar rubutu) - S2, S3, ..., Sn: Sannu a hankali fadada nuclei - Sn: Cikakken rubutu yankin ** Progressive Scaling Algorithm **: **1. Farawa **: - Fara tare da mafi ƙarancin S1 - Yi amfani da Connectivity Component Analysis don samun misalai na rubutu **2. Fadada Iterative **: Domin i a cikin kewayon (2, n + 1): Ga kowane misali na rubutu: Nemo pixels na kusa a cikin Si Haɗa pixels da ke kusa da su a cikin misali na yanzu Sabunta iyakokin misali **3. Sharuɗɗan ƙarewa **: - Isa matsakaicin sikelin Sn Ko kuma ba zai yiwu a ci gaba da fadada shi ba. ** Aikin Asarar **: ** Cikakken Asarar **: L = Σ (i = 1 zuwa n) λi * L_seg(Si, Gi) Daga cikin su: - L_seg: Split loss (Dice loss + cross-entropy loss) - Gi: Lakabin gaskiya na sikelin ith - λi: Nauyi a ma'auni daban-daban ### PixelLink algorithm PixelLink yana gano rubutu ta hanyar hango haɗi tsakanin pixels. ** Core Idea **: - Pixel classification: Ƙayyade idan kowane pixel yana cikin rubutu - Hasashen haɗi: Hasashen haɗi tsakanin pixels na kusa - Misali segmentation: Aggregate pixels to form text instances through connection relationships ** Tsarin cibiyar sadarwa **: **1. Hasashen Rubutu / Non-Rubutu **: - Ayyukan rarrabuwa na binary - Yiwuwar rubutun fitarwa a kowace pixel **2. Hasashen haɗi **: - Hasashen haɗi a cikin hanyoyi 8 - Yiwuwar haɗi na fitarwa don kowane shugabanci ** Post-Processing Algorithms **: **1. Pixel Filtering **: - Tace pixels dangane da yiwuwar rubutu - Riƙe pixels na rubutu mai ƙarfi **2. Haɗin Haɗin **: - Yi amfani da kuma bincika algorithms - Haɗa pixels dangane da alaƙar haɗi - Form connected text instances ## Kimanta ma'auni da ƙididdigar ƙi ### Kimanta ma'auni ** Alamun matakin ganowa **: - Daidaito: Gano madaidaicin rabo na yankunan rubutu - Tunawa: Adadin ainihin yankunan rubutu waɗanda aka gano daidai - F1 Score: Matsakaicin daidaito da tunawa ** Ma'aunin Matakin Pixel **: - Pixel Accuracy: Kashi na pixels da aka rarraba yadda ya kamata - Pixel Recall: Kashi na rubutu pixels da aka rarraba daidai - IoU: Rabo na yankin tsinkaya zuwa ainihin yanki ### Benchmark dataset ** Jerin ICDAR **: - ICDAR 2013: Focused Horizontal Text Detection - ICDAR 2015: Ya ƙunshi rubutun da yawa - ICDAR 2017: Multilingual Text Detection ** Sauran mahimman bayanai **: - MSRA-TD500: Multi-directional dogon rubutu Lines - COCO-Text: Rubutu a cikin al'amuran halitta - Total-Text: Curved text detection - CTW1500: Rubutun siffar da ba a yarda da shi ba ## Aikace-aikacen Aikace-aikacen Aikace-aikacen T ### Inganta Aiki ** Model Compression**: - Knowledge distillation: Koyi manyan samfura tare da ƙananan samfura - Model pruning: Cire haɗin da ba shi da mahimmanci - Quantization: Rage lambobi daidaito ** Inference Acceleration **: - TensorRT: NVIDIA GPU hanzarta - OpenVINO: Intel hardware optimization - Inganta Wayar hannu: An inganta shi don masu sarrafa ARM ### Dabarun Tura ** Girgije Tura **: - High-Precision Models: Yi amfani da hadaddun tsarin cibiyar sadarwa - Batch Processing: Increases throughput - Elastic Scaling: Ta atomatik sikelin dangane da lodi ** Edge Deployment **: - Nau'ikan nau'ikan nau' - Real-lokaci sarrafawa: Low latency bukatun - Works Offline: Babu haɗin intanet da ake buƙata ## Abubuwan da ke faruwa a nan gaba ### Jagoran Ci gaban Fasaha ** Multimodal Fusion **: - Haɗa Samfuran Harshe: Yin amfani da bayanan rubutu - Multi-firikwensin fusion: Ya haɗu da bayanai kamar zurfin, infrared, da dai sauransu - Bayanin lokaci: Yi amfani da dangantakar lokaci a cikin bidiyo ** Gyare-gyare na Adaptive **: - Yankin Daidaitawa: Daidaitawa zuwa yanayi daban-daban da rarraba bayanai - Less-Shot Learning: Daidaitawa da sauri zuwa sababbin nau'ikan rubutu - Koyon kan layi: Ci gaba da haɓakawa dangane da ra'ayoyin masu amfani ** Ƙarshen Ƙarshen Ƙarshe **: - Ganowa da ganewa mace: Haɗin gwiwa na ganowa da ganewa - Multitasking Learning: Shiga cikin ayyuka masu alaƙa da yawa a lokaci guda - Binciken gine-ginen neural: Tsara tsarin cibiyar sadarwa ta atomatik ### Fadada aikace-aikacen ** Abubuwan da ke tasowa **: - AR / VR: Gano rubutu a cikin gaskiyar da aka haɓaka - Tuki mai cin gashin kansa: Alamar zirga-zirgar ababen hawa da alamar titi - Gwajin Masana'antu: Lakabin samfur da sarrafa inganci ** Aikace-aikacen Cross-Domain **: - Medical Imaging: Rubutu daga bayanan likita da rahotanni - Remote Sensing Imagery: Gano sunan wuri a cikin hotunan tauraron dan adam - Tarihin Tarihi: Digitization na tsoffin littattafai da rubuce-rubuce ## Summary Gano rubutu, a matsayin mahimmin ɓangare na tsarin OCR, ya sami ci gaba mai mahimmanci a zamanin ilmantarwa mai zurfi. Daga ingantaccen ganowa na EAST zuwa ainihin lokacin sarrafawa na DBNet zuwa gano siffar PSENet, kowane algorithm yana da nasa fa'idodi na musamman da yanayin da ya dace. ** Mahimman Abubuwan Fasaha **: - Multi-Scale Feature Fusion: Yana riƙe da rubutu mai girma dabam dabam - Geometry Modeling: Yana tallafawa gano rubutu na siffofi marasa kyau - Ingantaccen Ƙarshen-zuwa-Ƙarshe: Sauƙaƙe tsarin tsara da tsarin horo - Real-Time Tunani: Daidaita buƙatar daidaito da ingantaccen ** Shawarwarin Zaɓi **: - Ba da fifiko ga daidaito: Zaɓi algorithms masu rikitarwa kamar PSENet - Ba da fifiko ga sauri: Zaɓi algorithms mara nauyi kamar DBNet - Versatility: Zaɓi algorithms kamar EAST wanda ke daidaita aiki Tare da ci gaba da haɓaka fasaha, algorithms na gano rubutu za su ci gaba da haɓaka cikin jagorancin mafi girman daidaito, saurin sauri, da ƙarfin ƙarfin gama gari, yana ba da tushe mai ƙarfi na fasaha don yaduwar aikace-aikacen tsarin OCR.