【Mmụta miri emi OCR Series · 6】 Nyocha miri emi nke ihe owuwu CRNN

## Okwu Mmalite CRNN (Convolutional Recurrent Neural Network) bụ otu n'ime ihe owuwu kachasị mkpa n'ọhịa nke mmụta miri emi OCR, nke Bai Xiang et al. tụrụ aro na 2015. CRNN ji amamihe jikọta ikike mmịpụta atụmatụ nke netwọk neural convolutional (CNNs) na ikike ịme ngosi usoro nke netwọk neural ugboro ugboro (RNNs) iji nweta njirimara ederede njedebe na njedebe. Isiokwu a ga-enye nyocha miri emi banyere nhazi ụlọ CRNN, ụkpụrụ ọrụ, usoro ọzụzụ, na ngwa akọwapụtara na OCR, na-enye ndị na-agụ akwụkwọ nghọta teknụzụ zuru oke. ## Nchịkọta nke CRNN Architecture ### Design Mkpali Tupu CRNN, usoro OCR na-anabatakarị usoro nzọụkwụ: a na-ebu ụzọ chọpụta agwa na nkewa, mgbe ahụ, a ghọtara agwa ọ bụla. Usoro a nwere nsogbu ndị a: * Mgbochi nke usoro ọdịnala **: - Njehie mgbasa ozi: Njehie na nkewa agwa nwere ike imetụta nsonaazụ mmata ozugbo - Mgbagwoju anya: Na-achọ ịmepụta usoro nkewa agwa dị mgbagwoju anya - Ogbenye robustness: Na-enwe mmetụta na agwa spacing na font mgbanwe - Enweghị ike ijikwa ọrịa strok na-aga n'ihu: Ihe omume nke ọrịa strok na-aga n'ihu na ederede ejiri aka sie ike ikewapụ ** CRNN's Innovative Ideas**: - Mmụta njedebe: Ịmepụta ihe oyiyi ozugbo site na ihe oyiyi gaa na usoro ederede - Enweghị nkewa: Na-ezere mgbagwoju anya nke agwa segmentation - Usoro nlereanya: Jiri RNNs mee ihe nlereanya ịdabere n'etiti ihe odide - CTC Alignment: Adreesị ntinye-mmepụta usoro ogologo mismatches ### N'ozuzu ụkpụrụ ụlọ Ihe owuwu CRNN nwere isi ihe atọ: **1. Convolutional Layers*: - Ọrụ: Wepụ atụmatụ usoro si input oyiyi - Input: Ederede akara oyiyi (ofu elu, variable obosara) - Mmepụta: Atụmatụ map usoro **2. Ugboro ugboro n'ígwé **: - Ọrụ: Model contextual dependencies in feature sequences - Input: The feature usoro wepụtara site CNN - Mmepụta: Usoro njirimara na ozi gbara ya gburugburu **3. Transcription Layer **: - Ọrụ: Tọghata atụmatụ usoro ka ederede usoro - Usoro: Iji CTC (Connectionist Temporal Classification) - Mmepụta: Nsonaazụ ude ederede ikpeazụ ## Nkọwa zuru ezu nke convolutional oyi akwa ### Atụmatụ mmịpụta atụmatụ A na-ahazi ihe oyi akwa convolutional nke CRNN kpọmkwem maka nghọta ederede: ** Network Structure Atụmatụ **: - Omimi: 7 n'ígwé nke convolutional n'ígwé na-ejikarị - Obere convolutional kernels: 3×3 convolutional kernels na-ejikarị eme ihe - Pooling Strategy: Jiri pooling sparingly na obosara ntụziaka ** Nhazi netwọk a kapịrị ọnụ **: Input: 32×W×1 (elu 32, obosara W, otu ọwa) Conv1: 64 3×3 convolutional nuclei, nzọụkwụ 1, jupụta 1 MaxPool1: 2×2 ọdọ mmiri, nzọụkwụ ogologo 2 Conv2: 128 3×3 convolutional kernels, nzọụkwụ 1, jupụta 1 MaxPool2: 2×2 pooled, nzọụkwụ size 2 Conv3: 256 3×3 convolutional nuclei, nzọụkwụ 1, jupụta 1 Conv4: 256 3×3 convolutional cores, nzọụkwụ 1, jupụta 1 MaxPool3: 2×1 pooled, nzọụkwụ size (2,1) Conv5: 512 3×3 convolutional cores, nzọụkwụ 1, jupụta 1 BatchNorm + ReLU Conv6: 512 3×3 convolutional kernels, nzọụkwụ 1, jupụta 1 BatchNorm + ReLU MaxPool4: 2×1 pooled, nzọụkwụ size (2,1) Conv7: 512 2×2 convolutional nuclei, nzọụkwụ 1, jupụta 0 Mmepụta: 512×1×W / 4 ### Key Design Considerations ** Atụmatụ mkpakọ dị elu **: - Ebumnuche: Mpikota onu na 1 pixel elu - Usoro: Jiri nwayọọ nwayọọ mpikota onu elu site na iji multiple pooling n'ígwé - Ihe kpatara ya: Ịdị elu nke akara ederede adịghị mkpa ** Width Holding Strategy **: Nkowasi: Jikwaa gị obosara ozi nke onyinyo dị ka o kwere mee. - Usoro: Belata pooling arụmọrụ na obosara direction - Ihe kpatara ya: A na-egosipụta ozi usoro nke ederede na ntụziaka obosara ** Ntughari Map **: A ghaghị ịgbanwe mmepụta nke convolutional oyi akwa na usoro ntinye nke RNN: - Raw mmepụta: C×H×W (ọwa × elu× obosara) - Gbanwere: W ×C (usoro ogologo× akụkụ atụmatụ) - Usoro: Were njirimara vector maka ọnọdụ obosara ọ bụla dị ka nzọụkwụ oge ## Nkọwa zuru ezu nke okirikiri okirikiri ### RNN Nhọrọ CRNNs na-ejikarị LSTM bidirectional dị ka oyi akwa loop: * Uru nke Bidirectional LSTM **: - Contextual Information: Jiri ma n'ihu na azụ - Ịdabere na Ogologo Anya: LSTM nwere ike ijikwa ịdabere na ogologo oge - Gradient Stabilization: Na-ezere nsogbu nke gradient disappearance ** Nhazi netwọk **: Input: W×512 (usoro ogologo × atụmatụ akụkụ) BiLSTM1: mkpụrụ ndụ 256 zoro ezo (128 n'ihu + 128 azụ) BiLSTM2: mkpụrụ ndụ 256 zoro ezo (128 n'ihu + 128 azụ) Mmepụta: W×256 (usoro ogologo× zoro ezo akụkụ) ### Usoro nhazi usoro ** Oge Dependency Modeling**: The RNN oyi akwa na-ejide oge dependencies n'etiti odide: - Ozi nke agwa gara aga na-enyere aka n'ịmata agwa dị ugbu a - Ozi maka ihe odide ndị na-esote nwekwara ike inye ọnọdụ bara uru - Ozi nke okwu ma ọ bụ ahịrịokwu dum na-enyere aka disambiguate ** Nkwalite atụmatụ **: Atụmatụ ndị a na-ahazi site na RNN nwere njirimara ndị a: - Context-sensitive: Njirimara nke ọnọdụ ọ bụla nwere ozi gbara ya gburugburu - Oge nkwekọrịta: Atụmatụ na ebe ndị dị nso nwere ụfọdụ ịga n'ihu - Semantic ọgaranya: Na-agwakọta visual na usoro atụmatụ ## Nkọwa zuru ezu nke usoro ihe omume ahụ ### CTC usoro CTC (Connectionist Temporal Classification) bụ akụkụ dị mkpa nke CRNN: * Ọrụ nke CTCs: - Addressing Alignment Issues: Input usoro ogologo adịghị dakọtara mmepụta usoro ogologo - Ọzụzụ ọgwụgwụ: Ọ dịghị mkpa maka agwa-larịị nhazi annotations - Handle duplicates: Jikwaa ikpe nke oyiri odide n'ụzọ ziri ezi ** Olee otú CTC si arụ ọrụ **: 1. Gbasaa labeelu setịpụrụ: Tinye blank labelụ n'elu mbụ agwa set 2. Ndepụta Ụzọ: Edepụta ụzọ niile enwere ike ịhazi 3. Ụzọ puru omume: gbakọọ ihe puru omume nke ọ bụla ụzọ 4. Marginalization: chịkọta ihe puru omume nke ụzọ niile iji nweta usoro nke puru omume ### CTC ọnwụ ọrụ ** Nnọchiteanya mgbakọ na mwepụ **: N'iburu n'uche usoro ntinye X na usoro ebumnuche Y, a na-akọwa ọnwụ CTC dị ka: L_CTC = -log P(Y| X) ebe P(Y| (b) A na-achịkọta ihe ndị na-eme ka ọ bụrụ na ị P (Y | X) = Σ_π∈B^(-1)(Y) P(π| X) N'ebe a, B ^ (-1) (Y) na-anọchite anya usoro ụzọ niile enwere ike ịmepụta na usoro ebumnuche Y. ** Aga n'ihu-azụ algọridim **: Iji gbakọọ ọnwụ CTC nke ọma, a na-eji algọridim na-aga n'ihu-azụ maka mmemme dị ike: Algorithm na-aga n'ihu: Na-agbakọ ohere nke iru steeti ọ bụla. - Backward Algorithm: Na-agbakọ ihe puru omume site na steeti ọ bụla ruo na njedebe - Gradient Calculation: Gbakọọ gradients na njikọ na n'ihu-azụ puru omume ## Atụmatụ Ọzụzụ CRNN ### Data preprocessing ** Nhazi ihe oyiyi **: - Size normalization: Mee ka onyinyo dị elu na 32 pikselụ - Akụkụ Ratio Maintenance: Na-echekwa akụkụ nke ihe oyiyi mbụ - Grayscale Akakabarede: tọghata ka a otu-ọwa grayscale oyiyi - Ọnụ ọgụgụ normalization: a na-edozi ụkpụrụ pixel na [0,1] ma ọ bụ [-1,1] ** Nkwalite data **: - Mgbanwe geometric: ntụgharị, ntụgharị, mgbanwe echiche - Mgbanwe ọkụ: nchapụta, mgbanwe ọdịiche - Mkpọtụ mkpọtụ: Mkpọtụ Gaussian, nnu na ose mkpọtụ - Blur: Motion blur, Gaussian blur ### Ọzụzụ Ọzụzụ ** Usoro mmụta **: - Ọnụego mmụta mbụ: A na-edokarị ya na 0.001 - Decay Strategy: Exponential decay or step decay - Usoro okpomọkụ: Oge ole na ole mbụ na-eji obere mmụta ** Usoro nhazi **: - Dropout: Tinye a dropout mgbe RNN oyi akwa - Ibu mmebi: L2 regularization na-egbochi overfitting - Batch normalization: Jiri ogbe normalization na CNN oyi akwa ** Nhọrọ Optimizer **: - Adam: Mgbanwe mmụta ọnụego, ngwa ngwa convergence - RMSprop: Kwesịrị ekwesị maka ọzụzụ RNN - SGD + Momentum: Nhọrọ ọdịnala ma kwụsie ike ## Njikarịcha na mmelite nke CRNN ### Njikarịcha Architecture ** CNN Ele mmadụ anya n'ihu mmelite **: - Njikọ ResNet: Agbakwunyere njikọ fọdụrụ iji melite nkwụsi ike ọzụzụ - DenseNet Fabric: Njikọ siri ike na-eme ka atụmatụ multiplexing dịkwuo mma - Usoro Nlebara Anya: Na-ewebata nlebara anya na CNNs ** RNN Partial Improvements**: - GRU nchigharị: Jiri GRU iji belata ọnụọgụ nke parameters - Transformer: Na-edochi RNNs site na iji usoro nlebara anya onwe onye - Multi-Scale Features: Tinye atụmatụ sitere na akpịrịkpa dị iche iche ### Njikarịcha arụmọrụ ** Inference Acceleration **: - Model Quantization: INT8 quantization na-ebelata mgbakọ na mwepụ mgbalị - Model pruning: Wepụ njikọ ndị na-adịghị mkpa - Ihe Ọmụma Distillation: Mụta ihe ọmụma nke nnukwu ụdị na obere ụdị ** Njikarịcha ebe nchekwa **: - Gradient checkpoints: Belata akara ụkwụ ebe nchekwa n'oge ọzụzụ - Mixed Precision: Train na FP16 - Dynamic graph njikarịcha: Ebuli ọdịdị nke eserese gbakọtara ## Real-World Ngwa Ikpe ### Ederede ederede ejiri aka mee ihe ** Ọnọdụ ngwa **: - Digitize handwritten notes - Mpempe akwụkwọ autofill - Akụkọ ihe mere eme akwụkwọ mmata ** Atụmatụ teknụzụ **: - Nnukwu ọdịiche dị iche iche: Na-achọ ikike mmịpụta atụmatụ siri ike - Usoro ọrịa strok na-aga n'ihu: Uru nke usoro CTC doro anya - Context Matters: RNNs 'usoro ịme ngosi uwe ikike dị oke mkpa ### E biri ebi ederede ude ** Ọnọdụ ngwa **: - Digitize akwụkwọ - Njirimara tiketi - Signage ude ** Atụmatụ teknụzụ **: - Font Regularity: CNN atụmatụ mmịpụta bụ dịtụ mfe - Iwu Typography: Enwere ike iji ozi nhazi mee ihe - High ziri ezi chọrọ: Chọrọ ezi nlereanya tuning ### Njirimara ederede ederede ** Ọnọdụ ngwa **: - Street View Text Recognition - Njirimara akara ngwaahịa - Akara ngosi okporo ụzọ ** Atụmatụ teknụzụ **: - Ndabere dị mgbagwoju anya: Na-achọ mmịpụta atụmatụ siri ike - Nrụrụ siri ike: A chọrọ imewe ihe owuwu siri ike - Real-Time Chọrọ: Na-achọ echiche dị mma ## Nchịkọta Dị ka ihe owuwu kpochapụwo nke OCR mmụta miri emi, CRNN na-edozi ọtụtụ nsogbu nke usoro OCR ọdịnala. Usoro ọzụzụ ya na njedebe, echiche imewe na-enweghị nkewa agwa, na iwebata usoro CTC niile na-enye mkpali dị mkpa maka mmepe nke teknụzụ OCR. ** Onyinye dị mkpa **: - Njedebe: Na-eme ka nhazi nke usoro OCR dị mfe - Usoro Modeling: Jiri usoro nke ederede eme ihe n'ụzọ dị irè - CTC Alignment: Edozi usoro ogologo mismatch Usoro dị mfe: Ọ dị mfe nghọta ma mejuputa ya ** Ntuziaka mmepe **: - Nlebara anya: Iwebata nlebara anya iji meziwanye arụmọrụ - Transformer: Na-edochi RNNs na nlebara anya onwe onye - Multimodal fusion: Jikọta ozi ndị ọzọ dị ka ụdị asụsụ - Fechaa imewe: nlereanya mkpakọ maka mobile ngwaọrụ Ihe ịga nke ọma nke CRNN bụ ihe akaebe maka nnukwu ikike nke mmụta miri emi n'ọhịa nke OCR ma na-enye ahụmịhe bara uru maka ịghọta otu esi emepụta usoro mmụta dị irè. N'isiokwu na-esonụ, anyị ga-eleba anya na mgbakọ na mwepụ na mmejuputa iwu nke CTC ọnwụ ọrụ.